How to prevent python playwright timing out when taking many screenshots

Question

I am trying to take multiple screenshots of a reddit thread using playwright. It works by starting at the first xpath of the first element of the thread and then increments that xpath till it reaches a non-existent xpath. It works for the first 27 screenshots but after that it times out.

import pickle
from playwright.sync_api import sync_playwright,Page
with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    context =browser.new_context()
    page = context.new_page()
    cookies = pickle.load(open("cookies.pkl", "rb"))
    context.add_cookies(cookies)
    #this is the link it goes to to begin taking screnshots
    page.goto('https://www.reddit.com/r/AskReddit/comments/11iwm33/who_is_a_bad_guy_in_history_who_actually_wasnt_a/')
    #reloads just to make sure there are no popups
    i=2
    page.reload()
    page.set_viewport_size({"width": 640, "height": 480})
    page.reload()
    page.reload()
    #this loop keeps going till a invalid xpath is found and screenshots it.
    while True:
        try:
            page.locator("xpath=/html/body/div[1]/div/div[2]/div[2]/div/div/div/div[2]/div[3]/div[1]/div[3]/div[6]/div/div/div/div[%s]"%i).screenshot(path='screenshots/'+(str(i)+'.png'))
            i+=1
        except Exception as e:
            print(e)
            break

I havent tried much because there seems to be little answers that fix the problem all though I believe that the reason why this occurs is that the javascript on the page is being executed so much it causes a timeout. Also as the screenshots get made they get slower and slower towards the exception that breaks the screenshot loop. I also tried writing the exact same program with Selenium and got the same results and I figured by changing librarys I may be able to get different results but I actually go the exact same error I was trying to run from.

This is the error

Timeout 29656ms exceeded.
=========================== logs ===========================
taking element screenshot
  waiting for element to be visible and stable
    element is not visible - waiting...
============================================================

Edit the issue may have been this because its trying to take a huge screenshot

Answer 1

A blank div caused this error to occur