Search code examples
javascriptnode.jspuppeteerwebautomation

Puppeteer Get all data attribute values


My html doc is

<div class="inner-column">
 <div data-thing="abc1"></div>
 <div data-thing="abc2"></div>
 <div data-thing="abc3"></div>
</div>

How can I get all "data-thing" value (eg. ["abc1", "abc2", "abc3"]) inside div with class .inner-column?

const puppeteer = require('puppeteer');
const fs = require('fs');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  page.setViewport({width: 1440, height: 1200})
  await page.goto('https://www.example.com')

  const data = await page.content();

  await browser.close();
})();

Solution

  • You could use the page.$$eval function for that like this:

    const dataValues = await page.$$eval(
        '.inner-column div',
        divs => divs.map(div => div.dataset.thing)
    );
    

    Explanation

    What the page.$$eval function does (quote from the docs linked above):

    This method runs Array.from(document.querySelectorAll(selector)) within the page and passes it as the first argument to pageFunction.

    If pageFunction returns a Promise, then page.$$eval would wait for the promise to resolve and return its value.

    Therefore, it will first query the targeted divs and then map the divs to their data-* value by using the dataset property.