Search code examples
javascriptweb-scrapingscreen-scrapingend-to-endpuppeteer

puppeteer page.evaluate path vs test-ID

Im trying get the value from a div in puppeteer e2e tests. The pice of html and js looks a bit like this. The problem is the results. When I run this code with the id's its result is literally ' ' just a empty line in the console.

However if I swap },'[data-testid="TestID10"]') with the commented line of code I get the right response 'Hello World' in the console. (note: using Full html selector path's is not a good practice when working with puppeteer)

Can anyone tell me why this won't work with ID's? and how I can get around this since using HTML selectors is really a bad practice.

Kind regards! 

const value1 = await page
        .waitForSelector('[data-testid="TestID10"]')
        .then(() => {
            return Promise.resolve(
                page.evaluate((path) => {
                    return window.document
                        .querySelector(path)
                        .value;
                },'[data-testid="TestID10"]')
            )
        });
    console.log(value1);
    
    //  },'#0-row-0 > div > div.TableCell.row-0.col-0.TextInputCell > div')
<div class="TextInputCell" data-testid="TestID10">
  <span class="value">Hello Word</span>
</div

Solution

  • The follwing works fine for me. Notice I put the selector into a variable rather than using a string. Also I am returning the innerText not value as in your example. HTH

    const puppeteer = require('puppeteer'); 

async function run() { 

const htmlStr = '<!doctype html><html><head></head>' +
  '<body><div class="TextInputCell" data-testid="TestID10">' +
  '<span class="value">Hello Word</span></div></body></html>';

  const browser = await puppeteer.launch({
      headless: false
  });
  const page = await browser.newPage();
  await page.setContent(htmlStr);    

  const selector = '[data-testid="TestID10"]';
  await page.waitForSelector(selector);

  var innerText = await page.evaluate((sel) => {
        return document.querySelector(sel).innerText;
  }, selector);

  console.log(innerText);

  browser.close();

};

run();