Selenium: Can't access cookie banner html

Question

I've been trying to access the html code of cookie banners using Selenium. For some websites, I can see the cookie banner html in the Firefox Web-Inspector, however, I cannot access it via Selenium.

For example https://faz.net. Here, driver.page_source does not contain the html code of the cookie banner and I also can't access it's elements via driver.find_elements (e.g. the "ZUSTIMMEN" - button. "zustimmen" means "to accept").

What I've tried so far:

    from selenium import webdriver
    driver = webdriver.Firefox()
    driver.implicitly_wait(20)
    driver.get("https://faz.net")
    print(driver.page_source)  # page source does not contain the button "ZUSTIMMEN"
    print(driver.find_elements_by_xpath('//button[text()="ZUSTIMMEN"]'))
    driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[text()=ZUSTIMMEN"]'))))

What am I doing wrong?

Answer 1

That button ZUSTIMMEN is in iframe. You need to switch the driver focus to iframe like below :

driver.get("https://faz.net")
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[id^='sp_message_iframe']")))
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='ZUSTIMMEN']"))).click()

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

once you are done with iframe, you can switch to default content like this :

driver.switch_to.default_content()