Search code examples
pythonscrapycss-selectorsplaywright-python

Playwright not loading all thumbnails on page


I'm using playwright with python. I have a pages with multiple thumbnails (>10), but everytime playwright is not detecting more than 8 with the css selector. Even though I manually checked and there are all the selectors available. Sample page : https://www.ah.nl/bonus/groep/-13928?week=37 Sample code:

        await page.wait_for_load_state()
        grid_count = await page.locator('article.product-card-portrait_root__ZiRpZ').count()
        products = page.locator('article.product-card-portrait_root__ZiRpZ')
        try:
            await expect(products.nth(grid_count)).to_be_visible()
        except:
            ipdb.set_trace() 

Every single time the page has more than 8 elements, it's going in the ipdb on the except. How can I fix this ?


Solution

  • It looks like there is some kind of lazy load, at first, first 8 elements are loaded, if you scroll down, more content will load