Search code examples
selenium-webdriverxpathwebdriverwebdriverwaitxpath-1.0

Webscraping with Selenium using Xpath

I'm looking for some numbers on https://tnnslive.com/

The numbers I'm looking for on tnnslive.com:

the numbers I'm looking for on tnnslive.com

If I copy the xpath in Chrome, I get:

//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]

If I run the following code:

from selenium import webdriver

# Start a new Chrome browser session
driver = webdriver.Chrome(executable_path="PATH_TO_CHROMEDRIVER")

# Navigate to the desired URL
driver.get('https://tnnslive.com/match/VDDGj17PodORnq62YwQk')

# Find the element by its XPath and print its text
element = driver.find_element("xpath", '//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]')    

# Close the browser
driver.quit()

the error occurs:

NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]"}
  (Session info: chrome=115.0.5790.171)

Why? How can I solve this issue?

Solution

  • To print the text associated with 1st Serve Points Won for the first player within the website you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

    • Using XPATH, following-sibling and text attribute:

      driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following-sibling::div[1]"))).text)

    • Using XPATH, following and get_attribute("innerHTML"):

      driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following::div[1]"))).get_attribute("innerHTML"))

    • Console output:

      64% (29/45)

    • Note : You have to add the following imports :

      from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

    You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

    References

    Link to useful documentation: