Search code examples
pythonhtmlselenium-webdriverclickelement

crawling web data selenium python


i tried to get content in this location. but there is only blank show. Please help me.
this is the elment: Đã bán 74k/tháng
This is my code.

print(driver.find_elements(By.CSS_SELECTOR, "div.r6HknA+uEPGHT"))

in this website:
https://shopee.vn/search?keyword=iphone&page=0&sortBy=sales enter image description here


Solution

  • To print the text Đã bán 74k/tháng ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

    • Using CSS_SELECTOR:

      driver.get("https://shopee.vn/search?keyword=iphone&page=0&sortBy=sales")
      print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.row.shopee-search-item-result__items > div > a div[data-sqe='rating'] +div"))).text)
      driver.quit()
      
    • Console Output:

      Đã bán 74k/tháng
      
    • Note : You have to add the following imports :

      from selenium.webdriver.support.ui import WebDriverWait
      from selenium.webdriver.common.by import By
      from selenium.webdriver.support import expected_conditions as EC
      

    You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python