Search code examples
pythonbeautifulsouppython-requests-html

Why index out of range?


i'm trying to webscrape some products and i'm getting this error whenever i'm enter the code

import bs4, requests

def getFravegaPrice(productUrl):
    res = requests.get(productUrl)
    res.raise_for_status ()

    soup = bs4.BeautifulSoup(res.text, 'html.parser')
    elems = soup.select('#wrapper > div.border-main > div > div > div.col-md-9.col-sm-9.col-xs-12 > div:nth-child(3) > ul > li:nth-child(7) > div')
    return elems[0].text
    
price = getFravegaPrice('https://compragamer.com/index.php?seccion=3&cate=62&nro_max=50')
print ('The price is ' + price) 

Solution

  • As already suggested, the selector you are using is incorrect, so elems is empty. Here is an alternative to scrape the price of an item.

    from bs4 import BeautifulSoup
    import requests
    
    res = requests.get('https://compragamer.com/index.php?seccion=3&cate=62&nro_max=50')
    res.raise_for_status()
    soup = BeautifulSoup(res.text, 'html.parser')
    item_name = "" #Enter item name you are scraping here
    for item in soup.find_all("a"):
        if item_name == item.text.strip():
            price = item.parent.parent.find("span", {"class": "products__price-new"}).text.strip()