Search code examples
javascriptnode.jspuppeteerwebautomation

Puppeteer does not change selector

I'm trying to automate the task of querying for data on this site using Puppeteer. So I need to select the dataset (Daily Summaries, 1st option), then select location type (State, 3rd option), then select state (Alaska, 2nd option). The problem is my code does not change to the next table. So instead of selecting the 3rd option (State) after selecting the 1st option in dataset (Daily Summaries), it just selects the 3rd option but in dataset table again! I am new to Puppeteer so I don't really know what to do with this. Any help is appreciated.

Below is my code:

const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({headless:false})
  const page = await browser.newPage()

  const navigationPromise = page.waitForNavigation()

  await page.goto('https://www.ncdc.noaa.gov/cdo-web/datatools/selectlocation')

  await page.waitForSelector('.selectLocationFilters > .datasetContainer > .slideElement > #datasetSelect > option:nth-child(1)')
  await page.click('.selectLocationFilters > .datasetContainer > .slideElement > #datasetSelect > option:nth-child(1)')

  await page.select('.inset #locationCategorySelect', '')

  await page.waitForSelector('.selectLocationFilters > .locationCategoryContainer > .locationCategoryFilter > #locationCategorySelect > option:nth-child(3)')
  await page.click('.selectLocationFilters > .locationCategoryContainer > .locationCategoryFilter > #locationCategorySelect > option:nth-child(3)')

  await page.select('.inset #selectedState', '')

  await page.waitForSelector('.selectLocationFilters > .locationContainer > .stateFilter > #selectedState > option:nth-child(2)')
  await page.click('.selectLocationFilters > .locationContainer > .stateFilter > #selectedState > option:nth-child(2)')

  await browser.close()
})()

This is what I want. Dataset -> Location type -> State Alaska. Instead the code keeps selecting only in the Dataset table. enter image description here

Solution

  • The problem you have there is that CSS transitions are preventing you from clicking those elements. One possible solution would be disabling all CSS animations on the page.

    You can add that after the goto call: 

    
await page.addStyleTag({ content : `
    *,
    *::after,
    *::before {
        transition-delay: 0s !important;
        transition-duration: 0s !important;
        animation-delay: -0.0001s !important;
        animation-duration: 0s !important;
        animation-play-state: paused !important;
        caret-color: transparent !important;
    }`})