Search code examples
python-3.xweb-scrapingbeautifulsouphtml-parsing

Why I can't scrape all the content within 'data-src' attribute of this HTML


I am trying to scrape all the data within the 'data-src' element of this html text:

[<div class="js-delayed-image-load" data-alt="A man covers his face during a sandstorm in Cairo, Egypt, 16 January 2019" data-height="549" data-src="https://ichef.bbci.co.uk/news/320/cpsprodpb/5DE9/production/_105214042_hi051682579.jpg" data-width="976"></div>,

, , , , , , , , , , , , , , , , , , , , ]

I am using this code:

image_containers = soup.find_all('div', class_ = 'js-delayed-image-load')
print(type(image_containers))
print(len(image_containers))

for image in image_containers:
    image.div['data-src']

And it is giving me this error:

TypeError                                 
Traceback (most recent call last)
<ipython-input-546-fa82366c888d> in <module>()
  4 image_containers
  5 for image in image_containers:
 ----> 6     image.div['data-src']

TypeError: 'NoneType' object is not subscriptable

Why is it giving me None? Can someone please tell me what am I doing wrong?

Thanks!


Solution

  • image is already a target div node. You don't need to extract div once again (it doesn't have child div so image.div returns None). Try

    for image in image_containers:
        image['data-src']