Loop through html tags

Question

I am trying to scrape a webpage using html parser and beautifulsoup. I am trying to get text from certain

tags. But since some of these have no texts at all so I get an attribute error for those which are empty. I am trying following code:

content = elements.find("p",{"class":"Text"}).text #Where elements is a bs4 tag inside a for loop 

After some iterations, I get following error:

AttributeError: 'NoneType' object has no attribute 'text'

Maybe I will have to try something like following:

while True:
    content = elements.find("p",{"class":"Text"}).text
    if type(content)==None:
        content = 'None'

But something is wrong in the above code

Answer 1

Before accessing text property of element you will have to check whether that element is not None.

while True:
    elem = elements.find("p",{"class":"Text"})
    if elem is not None:
        content = elem.text
    else:
        content = 'None'  # Any static value you want to give