Unable to Scrape Product Price from Amazon

Question

def scrapedPage(URL, user_agent):
    if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
        ssl._create_default_https_context = ssl._create_unverified_context

    header = {"User-Agent": user_agent}

    proxy = {'https': 'using-proxy'}

    page = requests.get(URL, headers=header, proxies=proxy)


    return page

I am not able to scrape the price of the product which is Rs 12,300

user_agent = random.choice(user_agent_list)

pricelink='https://www.amazon.in/gp/offer-listing/B07NXTH1BD/ref=dp_olp_afts?ie=UTF8&condition=all'

new_page = scrapedPage(pricelink, user_agent)

new_soup = BeautifulSoup(new_page.content, 'html.parser')

print(new_soup.prettify())


find_price = new_soup.find_all('span',attrs={'style':'text-decoration: inherit; white-space: nowrap'})

I tried many methods like findAll(), select(), find(), etc but I am not able to scrape the price for the same. Please help me out.

It is returning empty list

Answer 1

Male following changes in yout code:

new_soup = BeautifulSoup(new_page.content, 'lxml') # html.parser > lxml

find_price = new_soup.select('span.olpOfferPrice > span') # change selector
print(find_price[0].text)

Output

Rs. 12,300.00