Search code examples
pythonseleniumwebdriverwaitexpected-conditionlinktext

"TypeError: 'str' object is not callable" using WebDriverWait for link_text in Selenium through Python

This is my first post on Stack Overflow. I have been browsing and searching for every possible answer to this question on SO, and I figured at this point I should just ask a question, as I have been at this wall for days now. I am currently working on a web scraping project with Selenium in Python. I have already scraped one website and am currently on the second, and I have run into a seemingly intractable issue. The following code works perfectly fine with no errors on the page https://www.spd.de/standpunkte/:

try:
    driver.find_element_by_link_text("Familien").click()
except NoSuchElementException:
    WebDriverWait(driver, 12).until("Familien").click()

Meanwhile, the following code throws the error TypeError: 'str' object is not callable on the page https://www.spd.de/standpunkte/familie/:

try:
    driver.find_element_by_link_text("Mehr erfahren").click()
except NoSuchElementException:
    WebDriverWait(driver, 12).until("Mehr erfahren").click()

The error occurs on the line WebDriverWait(driver, 12).until("Mehr erfahren").click(). Based on my research up to now, I figure the specific issue is that something in this line is being interpreted as a str object, and is therefore unable to be called. There are, as I see it, two issues:

  1. I don't see any reason for this line to produce the error while the code on top, scraping the same website, but on a different page, works without issue. I checked and made sure there are no hidden elements on the page that are being clicked instead.

  2. I don't know what in the line is being treated as a str object. The whole line? Only part of it? If so, which part? Exacerbating this is that I can't really break down the line to see anything about it; both print(WebDriverWait(driver, 12).until("Mehr erfahren").click()) and print(type(WebDriverWait(driver, 12).until("Mehr erfahren").click())) just end up giving the error. I was able to determine that the .click() method is not the issue, as I tried setting the line to be equal to a variable, and then interacted with the variable, but that still produced the error on the same line as before; it never even got to the line with the .click() method.

Is there any way I can determine which part of the line is being interpreted as a str? Or is that even the issue at all?

This has been driving me crazy, so assistance would be greatly appreciated!

Solution

  • Your research was in the right direction.

    In your first usecase, the line within try was successful:

    driver.find_element_by_link_text("Familien").click()

    So the line within except was never called:

    WebDriverWait(driver, 12).until("Familien").click()

    Hence you don't see the error.

    In your second use-case, the line within try wasn't successful:

    driver.find_element_by_link_text("Familien").click()

    So the line within except was called:

    WebDriverWait(driver, 12).until("Mehr erfahren").click()

    which results into the error:

    TypeError: 'str' object is not callable

    Deep Dive

    WebDriverWait is invoked inconjunction with expected_conditions.

    You can find a couple of detailed discussions in:

    Now, element_to_be_clickable() should be called within a tuple as it is not a function but a class, where the initializer expects just 1 argument beyond the implicit self:

    class element_to_be_clickable(object):
    """ An Expectation for checking an element is visible and enabled such that you can click it."""
    def __init__(self, locator):
        self.locator = locator

    def __call__(self, driver):
        element = visibility_of_element_located(self.locator)(driver)
        if element and element.is_enabled():
            return element
        else:
            return False

    So instead of:

    WebDriverWait(driver, 12).until("Mehr erfahren").click()

    You need to (add an extra parentheses):

    WebDriverWait(driver, 12).until((EC.element_to_be_clickable(By.LINK_TEXT, "Mehr erfahren"))).click()

    You can find a detailed discussions in init() takes 2 positional arguments but 3 were given using WebDriverWait and expected_conditions as element_to_be_clickable with Selenium Python