Why I can't push data into an array inside block page.evaluate puppeteer crawler?

Question

var codes = [];    
await page.evaluate((codes) => {
    $(".offers-details").each(function (i, e) {
        if ($(e).find('.coupon-code').length > 0) {
            console.log($(e).find('.coupon-code .btn-copy').attr('data-clipboard-text'));
            codes.push($(e).find('.coupon-code .btn-copy').attr('data-clipboard-text'));
        }
    });
}, codes);

How to use jQuery outside page.evaluate() in handlePageFunction? I only want to use jQuery here.
It's not working when I use it in handlePageFunction. How to resolve it?

Answer 1

page.evaluate is a Puppeteer function that executes inside the browser, that's how you get to access the global $ variable (that is attached to the window variable).

While you can't use jQuery outside of the browser, you can use all the selectors functions from Puppeteer: https://pptr.dev/#?product=Puppeteer&version=v5.2.1&show=api-pageselector

Includes:

• page.$ for single elements (returns null if not found)
• page.$$ for multiple elements (always return an array)

You can manipulate/read/interact with the return types of those functions, in your case, would be something like this (not tested!):

  const offersDetails = await page.$$(".offers-details");
  const codes = [];

  for (const offerDetail of offersDetails) {
     const couponCode = await offerDetail.$('.coupon-code');

     if (couponCode) {
       const cpText = await couponCode.$eval('.btn-copy', (el) => el.attributes('data-clipboard-text'));
       console.log(cpText);
       codes.push(cpText);
     }
  }

notice that all page functions are async, and needs to be awaited.

A second way to do this is to return arrays from the page.evaluate, which is simpler.

const codes = await page.evaluate(() => {
   const codes = [];
   // code here
   return codes;
});