Search code examples
node.jsaws-lambdapuppeteerheadless

Puppeteer's Click API does not trigger on image map element

Puppeteer's Click API does not trigger on image map element.

I am using a puppeteer for scraping different e-commerce sites. Some e-commerce sites show a popup on page ready. I am trying to close that popup using click api by targeting element but somehow getting an error as "Node is either not visible or not an Html Element".
I have applied click on selectors:
coords='715,5,798,74'
#monetate_lightbox_mask'
body>div>div:nth-child(1)
body>div:nth-child(1):div:nth-child(1)
URLs for scraping:

https://www.hayneedle.com/product/humantouchijoymassageanywherecordlessportablemassager.cfm https://www.hayneedle.com/product/napoleonfiberglowventedgaslogset.cfm https://www.hayneedle.com/product/napoleonsquarepropanefirepittable1.cfm

Please suggest.
Regards,
Manjusha

Solution

  • I would personally use the following to wait for and click the close button:

    const close_button = await page.waitForSelector( '[id$="ltBoxMap"] > [href="#close"]' );
await close_button.click();

    But unfortunately, it appears that the website has implemented bot detection and is displaying the following page:

    enter image description here

    The source of the resulting web page looks like this:

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en-US"><head profile="http://gmpg.org/xfn/11">
        <meta http-equiv="content-type" content="text/html; charset=UTF-8">
        <meta name="viewport" content="width=1000">
        <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
        <meta http-equiv="cache-control" content="max-age=0">
        <meta http-equiv="cache-control" content="no-cache">
        <meta http-equiv="expires" content="0">
        <meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT">
        <meta http-equiv="pragma" content="no-cache">
        <title></title>
</head>
<body>
        <h1>Access To Website Blocked</h1>

</body></html>

    The bot detection service cannot be fooled simply by changing the user agent, so you will need to experiment with some other methods to bypass the service if you would like to scrape the website.