How do I accept the essential cookies from modal that is placed in shadow-root on a website using selenium and firefox?

Question

I am trying to us Selenium with Firefox so that it opens a new window and then clicks on the accept essential cookies button. Using the inspection mode, I get the following properties: inner html: Nur erforderliche Cookies zulassen outer html: Nur erforderliche Cookies zulassen CSS-Selector: button.btn:nth-child(1) CSS-Pfad: div#consent-layer.consent-layer div.consent-layer__btn-container button.btn.btn--secondary.js-accept-essential-cookies XPATH: /div/div[2]/button[1]

So far all of my tries did not work out. Here is an example:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Initialize the WebDriver using GeckoDriver
driver = webdriver.Firefox()

url = 'https://www.bahn.de/reisen/view/verbindung/muenchen/hof.shtml'

# Navigate to the webpage
driver.get(url)

# Wait for the button to be clickable
button_xpath = "//button[@class='btn btn--secondary js-accept-essential-cookies']"
button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, button_xpath))
)

# Click the button
button.click()

# Close the browser
driver.quit()

I also looked at other solutions of related topics, but they did not work for me.

Answer 1

Your button is placed inside shadow-root, to get internal shadow root structure, you should get it's host first and then get shadowRoot property.

Shadow host in current example is first element with tag div. (Doesn't seem to have unique attributes)

get_shadow_root function gets shadow root from host, using JS executor.

More about Shadow DOM

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
timeout = 10
wait = WebDriverWait(driver, timeout)

def get_shadow_root(element):
    return driver.execute_script('return arguments[0].shadowRoot', element)

driver.get("https://www.bahn.de/reisen/view/verbindung/muenchen/hof.shtml")
driver.maximize_window()

shadow_host = wait.until(EC.presence_of_element_located((By.TAG_NAME, 'div')))
shadow_container = get_shadow_root(shadow_host)
accept = WebDriverWait(shadow_container, timeout).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'button.js-accept-all-cookies')))
accept.click()
wait.until(EC.invisibility_of_element(accept))