Search code examples
pythonbeautifulsoupkeyerror

Python having the "self.attrs[key]]" error


I'm starting to learn code some complex things on python and today I decided to work with BeautifulSoup. The problem appear when I tried to get the title of a product, I attempt to change ".find" to ".findAll" and can't find the solution. Someone please help me. Here is my code:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as Soup
ListaSteam = "https://store.steampowered.com/search/?sort_by=Price_ASC&category1=998%2C996&category2=29"

#PAGINA - OBTENCION - CERRADA
Pagina = uReq(ListaSteam)
PaginaHtml = Pagina.read()
Pagina.close()

#1 PASO
PaginaSoup = Soup(PaginaHtml, "html.parser")
CodigoJuegos = PaginaSoup.find("div",{"id":"search_resultsRows"})
PRUEBA = CodigoJuegos.a.span["title"]
print(PRUEBA)

The error is as follows:

This is the error:
    `Traceback (most recent call last):
  File "C:\Users\Usuario\Desktop\******", line 14, in <module>
    PRUEBA = CodigoJuegos.a.span["title"]
  File "C:\Users\Usuario\AppData\Local\Programs\Python\Python39\lib\site-packages\bs4\element.py", line 1406, in __getitem__
    return self.attrs[key]
KeyError: 'title'

Solution

  • First of, you should use PEP8 styling. It is hard to read your code.

    If you want to solve it with the least amount of code change do the following:

    PRUEBA = CodigoJuegos.a.span.text
    

    That said, I scrape websites professionally using (among other tools bs4), and I'd go for something like this:

    import requests
    from bs4 import BeautifulSoup
    
    search_url = "https://store.steampowered.com/search"
    category1 = ('998', '996')
    category2 = '29'
    
    params = {
        'sort_by': 'Price_ASC',
        'category1': ','.join(category1),
        'category2': category2,
    }
    
    response = requests.get(
        search_url,
        params=params
    )
    
    soup = BeautifulSoup(response.text, "html.parser")
    elms = soup.find_all("span", {"class": "title"})
    
    for elm in elms: 
        print(elm.text)
    

    Output:

    Barro F
    The Last Hope: Trump vs Mafia - North Korea
    Ninja Stealth
    Tetropunk
    Oracle
    Legend of Himari
    Planes, Bullets and Vodka
    Shift
    Blast-off
    ...
    

    If you already have a dependency to bs4, you might as well also get requests too.