Search code examples
pythonseleniumselenium-webdrivertextgetattribute

How can I get text from an A tag with Selenium?

I have been trying to scrape some products online, but when I try and print the title from an A tag it gives me this output:

<selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

Now this is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd

PATH = "C:\Program Files (x86)\chromedriver.exe"

driver = webdriver.Chrome(PATH)
driver.get("https://egypt.souq.com")

dotd = "/html/body/div[2]/div/main/div[1]/div[1]/div/div[1]/a/img"

driver.find_element_by_xpath(dotd).click()

def get_deals():
    title_xpath = "/html/body/div[1]/div/main/div/div[4]/div[3]/div[2]/div[1]/div[1]/div/div[2]/ul/li[1]/h6/span/a"
    titles = driver.find_elements_by_xpath(title_xpath)
    for title in titles:
        print(title)

get_deals()
print("successful")
Solution

  • This output from print()...

    <selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

    ...isn’t any error, but the WebElement itself.

    It seems you were close. As you were able to extract the element, to extract the text within the element, you can use either of the following Locator Strategies:

    • Using text attribute:

      for title in titles:
    print(title.text)

    • Using get_attribute() attribute:

      for title in titles:
    print(title.get_attribute("innerHTML"))

    Reference

    You can find a couple of relevant discussions in: