Search code examples
pythonselenium-webdriverweb-scrapingcss-selectorswebdriverwait

In python/selenium, how to get Amazon web page item ratings information

In python/selenium, I want to get the item star in Amazon web page, but failed .Anyone can help ? Thanks!

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
chrome_options = webdriver.ChromeOptions()

chrome_options.add_experimental_option('excludeSwitches', ['enable-automation'])
chrome_options.add_experimental_option("detach", True)


prefs = {
            'profile.default_content_setting_values': {
                'images': 2,
            }
        }
chrome_options.add_experimental_option('prefs', prefs)

chrome_options.add_argument('lang=en')
driver = webdriver.Chrome(options=chrome_options)
driver.maximize_window()
driver.get('https://www.amazon.com/dp/B09VDD8D8R')
page_star = driver.find_element(By.CLASS_NAME,"a-icon-alt")
page_star_text = page_star.text
# the wished result of page_star_text is "4.1 out of 5 stars"

enter image description here

Solution

  • To get the rating text 4.1 out of 5 stars, Use WebDriverWait() and wait for presense of element located and since the node value is hidden instead .text use get_attribute("textContent")

    driver.get("https://www.amazon.com/dp/B09VDD8D8R")
print(WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR,"span.a-icon-alt"))).get_attribute("textContent"))

    You need to import below libraries

    from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

    console output:

    enter image description here