Django: Selenium- stale element reference on browse

Question

I am using Chrome webdriver, because the Firefox webdriver seems to have issues on my Windows PC.

My Django Functional Tests were working just fine before I went on vacation, but now they are throwing all kinds of errors.

Sometimes, when trying to find an element on a page I get:

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

Other times, an attempt to verify the url fails, because Selenium seems to be reading the URL from the previous page. The failure point seems to change from run to run, in other words, certain parts of the test can alternate between successful and unsuccessful between executions. However, the issue always seems to occur after using .click().

When watching the browser, Selenium seems to be navigating to the page successfully, so I suppose it is simply looking for items too quickly- before they exist in the browser.

Adding implicitly.wait() to my setUpClass doesn't seem to help, but I might be using it wrong.

I tried the promising idea (below) from Harry Percival's site, but Selenium simply times out while sitting on the page that it was told to wait for.

from contextlib import contextmanager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.expected_conditions import \
    staleness_of
class MySeleniumTest(SomeFunctionalTestClass):
    # assumes self.browser is a selenium webdriver

    @contextmanager
    def wait_for_page_load(self, timeout=30):
        old_page = self.browser.find_element_by_tag_name('html')
        yield
        WebDriverWait(self.browser, timeout).until(
            staleness_of(old_page)
        )

    def test_stuff(self):
        # example use
        with self.wait_for_page_load(timeout=10):
            self.browser.find_element_by_link_text('a link')
            # nice!

Has anyone else dealt with this? What is the proper approach to this problem that I should be following?

EDIT: The posted solution works brilliantly and really cleaned up my code, as I am able to simply add a .click() to the called function, just as described.

Here are the docs that helped me customize this:

Documentation for the syntax for By for modification purposes.

Documentation for Selenium's Expected Conditions

Note: I was using the term "browser" in place of driver, I think that is what was initially throwing me off.

Answer 1

Does wait_for_page_load have to be a generator? I think it would return without waiting! That's why your tests are flaky. Sometimes the element would have loaded by the time you call find_element_by_link_text sometimes not.

I have used this approach with success:

from selenium.webdriver.support import expected_conditions as EC

def wait_for_element(self, elm, by = 'id', timeout=10) :
    wait = WebDriverWait(self.driver, timeout)
    if by == 'id' :
        element = wait.until(EC.element_to_be_clickable((By.ID,elm)))
        return self.driver.find_element_by_id(elm)
    elif by == 'link':
        wait.until(EC.element_to_be_clickable( (By.LINK_TEXT,elm)))
        return self.driver.find_element_by_link_text(elm)
    # by tag, by css etc etc goes here.

I call this method with a prominent id of a dom element that should show up before the page can be interacted with. The returned element can be interacted with directly.