Find element in shadow DOM for python selenium when return arguments[0].shadowRoot doesn't seem viable

Question

I'm trying to click the accept cookies button in a python selenium session but it's inside a shadow DOM. Here is an outline of my code:

# Python version 3.10.9
# selenium version 3.141.0

from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep

url = "https://www.cottages.com/search?adult=2&child=0&infant=0&pets=0&range=3&nights=7&accommodationType=cottages&start=29-12-2023&page=1&sort=priceasc&regionId=21996&regionName=South+West+England"

driver = webdriver.Chrome(executable_path=r"C:/xxx/chromedriver-win64/chromedriver.exe")

driver.get(url)
sleep(5)
driver.implicitly_wait(5)
host = driver.find_element(By.XPATH, "/html/body/div[7]")
root = driver.execute_script("return arguments[0].shadowRoot", host)
root.find_element(By.CLASS_NAME , 'sc-dcJsrY jLOXfK').click()

This returned an AttributeError that dict has no attribute 'find_element'.

I had seen the approach elsewhere on here (e.g. How to handle elements inside Shadow DOM from Selenium ) which is why I tried it.

When I printed root, I saw that it was just the following dictionary:

{'shadow-6066-11e4-a52e-4f735466cecf': '92E369CF463093341CE4281E8449D891_element_93'}

Any ideas as to how to access this element to click it?

Answer 1

You should wait at least for presence of shadow host. Not sure why are you getting error AttributeError - I get error that element with locator sc-dcJsrY jLOXfK hasn't been found with your code.

However, this is general consent element and it can be accepted by code below:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
timeout = 10
wait = WebDriverWait(driver, timeout)

def get_shadow_root(element):
    return driver.execute_script('return arguments[0].shadowRoot', element)

driver.get("https://www.cottages.com/search?adult=2&child=0&infant=0&pets=0&range=3&nights=7&accommodationType=cottages&start=29-12-2023&page=1&sort=priceasc&regionId=21996&regionName=South+West+England")
driver.maximize_window()

shadow_host = wait.until(EC.presence_of_element_located((By.ID, 'usercentrics-root')))
shadow_container = get_shadow_root(shadow_host).find_element(By.CSS_SELECTOR, '[data-testid=uc-app-container]')
WebDriverWait(shadow_container, timeout).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '[data-testid=uc-accept-all-button]'))).click()
wait.until(EC.invisibility_of_element_located((By.ID, 'usercentrics-root')))

as an alternative approach you can get internal element via JS

shadow_container = driver.execute_script("return arguments[0].shadowRoot.querySelector('[data-testid=uc-app-container]')", shadow_host)

Another question with this consent component reference