Search code examples
javascriptpuppeteerwebautomation

Accessing child elements in puppeteer


I have the below HTML structure

<div class ="container" id= "12">
    <div class="details" desc-type= "multiline">
        <a href="#">
            <div class="description"> Some Description </div>
        </a>
    </div>
</div>

And I scraping this using the below code

const SELECTOR =
    "div.container";

const movies = await page.$$eval(
    SELECTOR,
      nodes =>
        nodes.map(element => {
          return {
            movieID: element.getAttribute("id"),
          };
        } )    
    );

How can I modify the above code so that I can read desc-type= "multiline" and innerText of <div class="description">?


Solution

  • How about this?

    const movies = await page.$$eval(
        SELECTOR,
          nodes =>
            nodes.map(element => {
              return {
                movieID: element.getAttribute("id"),
                descType: element.querySelector('[desc-type]').getAttribute('desc-type'), 
                description: element.querySelector(".description").innerText
              };
            } )    
        );