Search code examples
rrselenium

How do I create a clickable webElement from an elementID in a shadow DOM using RSelenium?

I'm trying download data from this GIS site, but some of the button elements I need are in a shadow dom.

enter image description here

My strategy was to run a JS script that uses shadowRoot.querySelector to get the "elementID" and use it to point to the button I need to click. I thought I could use RSelenium::webElement to create the button elements, but I'm getting an error when I try to click on them.

pacman::p_load(RSelenium, glue, dplyr, rvest)

driver <- rsDriver(browser = c("chrome"), chromever = "90.0.4430.24")
chrome <- driver$client

wisc_url <- "https://data.dhsgis.wi.gov/datasets/wi-dhs::covid-19-historical-data-by-county/about"

chrome$navigate(url = wisc_url)
Sys.sleep(5)

# open side panel
wisc_dl_panel_button <- chrome$findElement("css selector", "#main-region > div.content-hero > div.content-hero-footer > div.content-footer.flex-row > div.content-footer-right > div.yielded > button")
wisc_dl_panel_button$clickElement()
# sometimes it needs time to create the file
Sys.sleep(120)

# get elementId from shadow DOM
wisc_dlopts_elt_id <- chrome$executeScript("return document.querySelector('hub-download-card').shadowRoot.querySelector('calcite-card').querySelector('calcite-dropdown')")[[1]]
wisc_dlopts_elt <- webElement(elementId = wisc_dlopts_elt_id,
                              browserName = "chrome",
                              port = 4567)
wisc_dlopts_elt$clickElement()

## Error in checkError(res) : 
##  Undefined error in httr call. httr output: length(url) == 1 is not TRUE

I don't think it's the particular elementID that I'm using. I tried this create-webElement method with other elements that were able to be clicked using the typical findElement >> clickElement method, and I still get the same error.

BTW I can solve this particular problem through a JS script. But I don't know JS that well, so I'd rather have a RSelenium solution that I can more generally apply if different use cases pop-up in the future.

Solution

  • Use find_elements from the shadowr package:

    library(shadowr)
remDr <- RSelenium::remoteDriver(
  remoteServerAddr = "host.docker.internal",
  port = 4445 , browser = "chrome")
remDr$open(silent = TRUE)

wisc_url <- "https://data.dhsgis.wi.gov/datasets/wi-dhs::covid-19-vaccination-data-by-county/explore?location=44.718554%2C-89.846850%2C7.23"
remDr$navigate(url = wisc_url)
Sys.sleep(5)

shadow_rd <- shadow(remDr)

wisc_dl_panel_button <- find_elements(shadow_rd, 'button[aria-describedby*="tooltip"]')
wisc_dl_panel_button[[3]]$clickElement()

wisc_dl_panel_button <- find_elements(shadow_rd, 'calcite-button')
wisc_dl_panel_button[[1]]$clickElement()

    For details and examples please check out my github page https://github.com/ricilandolt/shadowr and https://www.shadowr.org/ (Disclosure: I'm a contributor to this package).