Selenium web driver cannot find button

Question

I am making a Python 3 script with Selenium 4 and Firefox Gecko web driver.

The goal is simple:

• Open a webpage
• Click on the blue "Greit"-button for accepting cookies

This is my Python 3 script so far, but it does not work as intended. The script does not seem to find the button. Perhaps because it is not part of the initial DOM. It is triggered by a javascript popup.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
import time

URL = 'https://examplepage.com'

driver = webdriver.Firefox()
driver.get(URL)

time.sleep(2)

print(driver.page_source)

wait = WebDriverWait(driver, 5)
wait.until(ec.visibility_of_element_located((By.XPATH, '//button[text()="Greit"]')))
driver.find_element(By.XPATH, '//button[text()="Greit"]').click()

driver.quit()

What can I do to make selenium find the button and click on it? The HTML for the popup is available in DevTools inspection, but not in the page source.

Answer 1

The element you trying to click is inside an iframe.
So you first have to switch to that iframe and then click the element.
This should work:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
import time

wait = WebDriverWait(driver, 10)

wait.until(ec.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(@id,'sp_message')]")))

wait.until(ec.visibility_of_element_located((By.XPATH, "//button[@title='Greit']"))).click()

driver.quit()

BTW you can set a wait timeout to be larger, it is usually set to 20 or 30 seconds. There is no problem using these timeouts since once an element is found Selenium will immediately continue to the next command. On the other hand, you don't want your tests to fail because of slow internet issues.
Also, you don't need to get the element again after the utilizing of wait. This method returns the web element, so you can use it as I show in my code.