Can't locate and click on a terms of conditions button

Question

I am new to pupeeteer and first what i am trying to do is loading a page and clicking on a button. However, it can't locate the element. I assume this is because I need to locate the parent or parent's parent element.

<button role="button" data-testid="uc-accept-all-button" class="sc-gsDKAQ fHGlTM" style="border: 2px solid rgb(247, 196, 0); padding: 0.375rem 1.125rem; margin: 0px 6px;">Accept All</button>

This is the full css selector taken from inspect

#focus-lock-id > div.sc-furwcr.lhriHG > div > 
div.sc-bYoBSM.egarKh > div > div > div.sc-dlVxhl.bEDIID > 
div > button:nth-child(3)

Here's my code:

const puppeteer = require("puppeteer");

async function launch() {
  const browser = await puppeteer.launch({
    headless: false,
    defaultViewport: false,
  });
  const page = await browser.newPage();

  await page
    .goto("", {
      waitUntil: "networkidle0",
    })
    .catch((err) => console.log("error loading url", err));
  page.click('button[data-testid="uc-deny-all-button"]');
}
launch();

It's a simple accept and conditions block where I would want to click on an "Accept all" button to close it and proceed further. What usual actions do I need to wait for the parent element first then dig further? Or there is an easy way?

This is the website I am trying to close terms and conditions for: https://www.partslink24.com/

Answer 1

A few problems:

1. The selector appears dynamically after the page has loaded, so you should waitForSelector rather than assuming networkidle0 will be enough to catch the button.
2. The selector you want is in a shadow root, so select the root and dip into it with .shadowRoot.
3. Your .click() call must be awaited.

const puppeteer = require("puppeteer"); // ^19.7.2

let browser;
(async () => {
  browser = await puppeteer.launch({headless: true});
  const [page] = await browser.pages();
  const url = "https://www.partslink24.com/";
  await page.goto(url, {waitUntil: "domcontentloaded"});
  const rootSel = "#usercentrics-root";
  const btnSel = 'button[data-testid="uc-deny-all-button"]';
  const rootContainer = await page.waitForSelector(rootSel);
  const root = await rootContainer.evaluateHandle(el => el.shadowRoot);
  const btn = await root.waitForSelector(btnSel);
  await btn.click();
  await page.waitForFunction(`
    !document.querySelector("${rootSel}")
      .shadowRoot
      .querySelector('${btnSel}')
  `);
  await page.screenshot({path: "test.png", fullPage: true});
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

An even shorter way is to use pierce/ to automatically query within open shadow roots:

// ...
  const btn = await page.waitForSelector("pierce/ " + btnSel);
  await btn.click();
// ...

See also Popup form visible, but html code missing in Puppeteer which has the same #usercentrics-root and offers other suggestions for getting rid of the popup.

---

Since the original post, the site has changed, as has the Puppeteer shadow DOM API.

The new interface is confusing, arguably using a dark pattern. There's a toggle to tick to not sell your personal information, but it's not clear which side corresponds to which setting. I assume ticking it on opts out.

From a Puppeteer perspective, there's now >>> which traverses the shadow DOM, replacing pierce/.

const puppeteer = require("puppeteer"); // ^19.11.1

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const url = "https://www.partslink24.com/";
  await page.goto(url, {waitUntil: "domcontentloaded"});
  const rootSel = "#usercentrics-root";
  const toggleSel = "#toggle-toggle-ccpa-banner";
  const btnSel = '[data-testid="uc-ccpa-button"]';
  const toggle = await page.waitForSelector(">>> " + toggleSel);
  await toggle.click();
  const btn = await page.waitForSelector(">>> " + btnSel);
  await btn.click();
  await page.waitForFunction(`
    !document.querySelector("${rootSel}")
      .shadowRoot
      .querySelector('${btnSel}')
  `);
  await page.screenshot({path: "test.png", fullPage: true});
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());