Search code examples
pythonseleniumselenium-webdriverweb-scrapingwebdriver

Iteratively reading a specific element from a <table> with Selenium for Python


I am trying to read in information from this table that changes periodically. The HTML looks like this:

<table class="the_table_im_reading">
  <thead>...</thead>
  <tbody>
    <tr id="uc_6042339">
      <td class="expansion">...</td>
      <td>
        <div id="card_6042339_68587" class="cb">
          <a href="/uniquelink" class="cl" onmouseover="cardHover('somecard');" onmouseout="cardOut()">TEXT I NEED TO READ</a>
      </td>
      <td>...</td>
      more td's
    </tr>
    <tr id="uc_6194934">...</tr>
      <td class="expansion">...</td>
      similar as the first <tr id="uc...">

I was able to get to the table using:

table_xpath = "//*[@id="content-wrapper"]/div[5]/table"
table_element = driver.find_element_by_xpath(table_xpath)

And I am trying to read the TEXT I NEED TO READ part for each unique <tr id="uc_unique number">. The id=uc_unique number changes periodically, so I cannot use find element by id.

Is there a way reach that element and read that specific text?


Solution

  • Looks like you can search via the anchor-element link (href-attribute), since I guess this will not change.

    via xpath:

    yourText = table_element.find_element_by_xpath(.//a[@href='/blahsomelink']).text
    

    UPDATE

    OP mentioned that his link is also changing (with each call?), which means that the first approach is not for him.

    if you want the text of the first row-element you can try this:

    yourText = table_element.find_element_by_xpath(.//tr[1]//a[@class='cl']).text
    

    if you know for example that the link element is always in the second data-element of the first row and there is only one link-element, then you can do this:

    yourText = table_element.find_element_by_xpath(.//tr[1]/td[2]//a).text
    

    Unless you provide more detailed requirements as to what you are really searching for, this will have to suffice so far...

    Another UPDATE

    OP gave more info regarding his requirement:

    I am trying to get the text in each row.

    Given there is only one anchor-element with class cl in each tr element you can do the following:

    elements = table_element.find_elements_by_xpath(.//tr//a[@class='cl'])
    for element in elements:
        row_text = element.text
    

    Now you can do whatever you need with all these texts...