Python (Selenium) Error: stale element reference: element is not attached to the page document

Question

I'm getting the stale element reference: element is not attached to the page document error from my code below. I see most examples say that its a time delay that would fix it but no luck.

Please ignore the amount of modules I have imported, I always just import everything when I come to do a web scraping job.

What I want to do is for each in-play game on William hills website, I want to click into each game, then collect the stats that are in the iframe at the top of the page - collecting this data wont be a problem, the mechanism to click into each game and come back out of it is what is causing the error. The code below clicks into the first one no problem, then backs out of that page but errors when it comes to click into the second game with the class name I have chosen below, does anybody know what I am doing wrong here?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import pyodbc
import datetime
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.headless = False
options.add_argument("--start-maximized")

driver_path = 'C:/Python39/chromedriver.exe'
driver = webdriver.Chrome(options=options, executable_path=driver_path)

driver.get('https://sports.williamhill.com/betting/en-gb/in-play/football')
time.sleep(2)
driver.find_element(By.CLASS_NAME,"cookie-disclaimer__button").click()

#define elements with css selector
elements = driver.find_elements(By.CSS_SELECTOR,'.btmarket__link-name.btmarket__link-name--ellipsis.show-for-desktop-medium')

#run through each element in elements variable
for element in elements:
    time.sleep(2)
    element.click()
    time.sleep(2)
    #MY CODE WILL GO HERE TO PULL THE DATA
    driver.back()
    time.sleep(2)

#stop browser from closing
input()

Answer 1

Stale element reference exception implies that an element that was earlier present does not exist anymore on the page.

In the Code while iterating over elements, you are performing a back() action, which might refresh the page. Due to this elements is no longer recognized by Selenium.

We can redefine the elements inside the for loop again, so that Selenium finds elements again.

Try with below code once.

elements = driver.find_elements(By.CSS_SELECTOR,'.btmarket__link-name.btmarket__link-name--ellipsis.show-for-desktop-medium')
# Get the Length of the list to iterate over
length = len(elements)

for i in range(length):
    time.sleep(2)
    # Need to redefine since performing back action refreshes the pages
    elements = driver.find_elements(By.CSS_SELECTOR,'.btmarket__link-name.btmarket__link-name--ellipsis.show-for-desktop-medium')
    elements[i].click()
    time.sleep(2)
    driver.back()
    time.sleep(2)