Search code examples
pythonseleniumweb-scrapingselenium-chromedriverselenium-webdriver-python

Is there a better way to fetch text from HTML table using selenium?

I've been trying to fetch the text circled in the attached image below.

Table Image

Website URL

My Code:

driver.find_element_by_xpath('/html/body/chrome/div/mat-sidenav-container/mat-sidenav-content/div/ng-component/entity-v2/page-layout/div/div/div/page-centered-layout[3]/div/div/div[1]/row-card[1]/profile-section/section-card/mat-card/div[2]/div/list-card/div/table/tbody/tr/td[2]/field-formatter/identifier-formatter/a/div/div')

And below is the output of my code:

NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/chrome/div/mat-sidenav-container/mat-sidenav-content/div/ng-component/entity-v2/page-layout/div/div/div/page-centered-layout[3]/div/div/div[1]/row-card[1]/profile-section/section-card/mat-card/div[2]/div/list-card/div/table/tbody/tr/td[2]/field-formatter/identifier-formatter/a/div/div"}

(Session info: chrome=89.0.4389.82).

Please how can I resolve this?

Solution

  • To get the value from dynamic table use WebDriverWait() and wait for visibility_of_all_elements_located() and following xpath.

    driver.get("https://www.crunchbase.com/organization/climeon/company_financials")
columnRecords=WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//h2[.='Funding Rounds']/following ::table[1]//tbody//tr//td")))

for col in columnRecords:
    print(col.text)

    You need to import below libraries

    from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC