Search code examples
node.jsweb-scrapinggoogle-cloud-platformpuppeteergoogle-datalayer

List element of a dataLayer with puppeteer


i'm doing a web scraping bot thanks to puppeteer, and i want to verify a field of a dataLayer on some url thanks to cloud functions on Google Cloud Plateform.

right now i'm using the await page.$('dataLayer') and it return a "true", so it find the dataLayer.

there is the dataLayer on my browser (not puppeteer) dataLayer

my code:

exports.datalayer = async(page) => {
    //verfify if there is a dataLayer
    let dl = await page.$('dataLayer')
    if(dl =! null){
        console.log('DataLayer found')
        
        //want to do something like that 
        for(let key in dl){
            let value = dl[key]
            console.log('key: ' + key + ' value: ' + value)
        }
        //or like that
        dl.forEach(element => {
            console.log(element)
        });

    }else{
        console.log('error with the dataLayer')
    }
}

In order to catch the OnetrustActiveGroups data.

when i do the forEach method i get this error:

log

and for the for method i get this error:

log2


Solution

  • better use the page.evaluate('dataLayer') then stringify it or use it as you want :)

    and with the evaluate the for(let item in dl) works fine !