Search code examples
pythonselenium-webdriverweb-scrapingxpathcss-selectors

Can't get element with Python Selenium find_element(By.CSS_SELECTOR)


I'm having trouble getting a specific element in the page I want to scrape. The content is wrapped in a weird tag, and not sure if it's an iframe. I also tried with CSS_SELECTOR, but anything coming after the mentioned tag throws an error.

The page I want to scrape: https://connect.echobotsales.de/#/company/YwOwE8kYmN

what I'm doing:

try:
    telephone_element = driver.find_element(By.XPATH, "//dynamicelements/div[2]/div[2]/div[2]/div[2]/span/a").text
    print(telephone_element)
except Exception as e:
    print("error", e)

The error I get:

error Message: no such element: Unable to locate element: {"method":"xpath","selector":"//dynamicelements/div[2]/div[2]/div[2]/div[2]/span/a"}
  (Session info: headless chrome=113.0.5672.63); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

Solution

  • Check the code below:

    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    driver = webdriver.Chrome()
    driver.get("https://connect.echobotsales.de/#/company/YwOwE8kYmN")
    
    wait = WebDriverWait(driver, 10)
    element = wait.until(EC.element_to_be_clickable((By.XPATH, "//dynamicelements/div[2]/div[2]/div[2]/div[2]/span/a")))
    print(element.text)
    print(element.get_attribute("innerText"))
    

    Console output:

    +49 89996400
    +49 89996400
    
    Process finished with exit code 0
    

    Another option: Although the above used XPath expression is correct, you can try using the below XPath expression also to locate the desired element. Refer the below line of code:

    element = wait.until(EC.element_to_be_clickable((By.XPATH, "//span//a[contains(@href,'tel')]")))