Search code examples
pythonscriptingnonetype

Python 3 'NoneType' object has no attribute 'text'

# import libraries
from urllib.request import urlopen
from bs4 import BeautifulSoup

#specify the url
html = 'https://www.bloomberg.com/quote/SPX:IND'

# query the website and return the html to thevariable 'page'
page = urlopen(html)

# parse the html using beautiful soup and store in variable 'soup'
data = BeautifulSoup(page, 'html.parser')

#take out the <div> of name and get its value
name_box = data.find('h1', attrs={'class': 'companyName_99a4824b'})

name = name_box.text.strip() #strip is used to remove starting and trailing
print (name)

# get the index price
price_box = data.find('div', attrs={'class':'priceText_1853e8a5'})
price = price_box.text
print (price)

I was following a guide on medium.com here and was having some conflictions due to lacking of knowledge of python and scripting, but I think I have my error at

name = name_box.text

because text is not defined and I am unsure they would like me to define it using the BeautifulSoup library. Any help maybe appreciated. The actual error will be below

 RESTART: C:/Users/Parsons PC/AppData/Local/Programs/Python/Python36-32/projects/Scripts/S&P 500 website scraper/main.py 
Traceback (most recent call last):
  File "C:/Users/Parsons PC/AppData/Local/Programs/Python/Python36-32/projects/Scripts/S&P 500 website scraper/main.py", line 17, in <module>
    name = name_box.text.strip() #strip is used to remove starting and trailing
AttributeError: 'NoneType' object has no attribute 'text'
Solution

  • The website https://www.bloomberg.com/quote/SPX:IND does not contain a <h1> tag with the class name companyName_99a4824b. That's why you are receiving the above error.

    In the website. <h1> tag look like this,

    <h1 class="companyName__99a4824b">S&amp;P 500 Index</h1>

    So to select it, you have to change the class name to companyName__99a4824b. 

    name_box = data.find('h1', attrs={'class': 'companyName__99a4824b'})

    Finally Result:

    # import libraries
from urllib.request import urlopen
from bs4 import BeautifulSoup

#specify the url
html = 'https://www.bloomberg.com/quote/SPX:IND'

# query the website and return the html to thevariable 'page'
page = urlopen(html)

# parse the html using beautiful soup and store in variable 'soup'
data = BeautifulSoup(page, 'html.parser')

#take out the <div> of name and get its value
name_box = data.find('h1', attrs={'class': 'companyName__99a4824b'}) #edited companyName_99a4824b -> companyName__99a4824b

name = name_box.text.strip() #strip is used to remove starting and trailing
print (name)

# get the index price
price_box = data.find('div', attrs={'class':'priceText__1853e8a5'}) #edited priceText_1853e8a5 -> priceText__1853e8a5
price = price_box.text
print (price)

    It would be better if you can also handle this exception, for future class name changes.