Need to scroll down in Dynamically Generated Webpage using VBA / Selenium

I am using Selenium to screen scrape a dynamically generated webpage. The trick is that the webpage does not appear to be generated until I manually scroll down the page. If I search for objects that are below the current screen when I open the page, I get an error saying the object (byClass or by XPath) don't exist. If I inspect the count of a occurrence of a multi-repeating class, it returns only a fraction of the total number. However, if I manually scroll down the page and the new content displays, then I can find the desired object using either byClass or byXPath, and the count grows, down to what has (ever) been displayed.

I have read through all the other posts with similar questions, but their solutions are not working in my case. The following has no impact on my page:

ch.ExecuteScript ("window.scrollTo(0, document.body.scrollHeight);")

Once I have (ever) scrolled down and back up the page manually, I can scroll to an object using

ch.FindElementByClass("css-w166kv-LegendLabel-LegendClickable").ScrollIntoView

But again, when I open the page, the instance of the class hasn't been generated yet, so at that time, I get the error that the class is not found.

I tried the following code (adapted from Python that I found posted in one of the other articles) that basically issues the same ExecuteScript as above, but tries to run it over and over, until everything is displayed. However, the height never changes, so it doesn't scroll.

Sub ScrollDownPage(ch As Selenium.ChromeDriver)

    Dim last_height As Long
    
'     # Get scroll height.
    last_height = ch.ExecuteScript("return document.body.scrollHeight")

    Do While True

'        # Scroll down to the bottom.
        ch.ExecuteScript ("window.scrollTo(0, document.body.scrollHeight);")

'        # Wait to load the page.
        Application.Wait (Now + TimeValue("0:00:02"))

'        # Calculate new scroll height and compare with last scroll height.
        new_height = ch.ExecuteScript("return document.body.scrollHeight")

        If new_height = last_height Then
            Exit Do
        End If
        
        last_height = new_height
    Loop

End Sub

My current work-around is that my screenscraping code will work if I make my zoom on my browser = 50% so that everything displays on screen.

ch.ExecuteScript ("document.body.style.zoom = '0.5'")

Any suggestions other than my Zoom Out hack?

Solution

I couldn't get the scroll to work properly, but I did find a workaround. My problem was the objects were not being generated until they were displayed. Instead of scrolling to have them displayed, I simply changed the Zoom level with Selenium to have the entire page displayed. It makes it so small it is pretty much unreadable visually, but it is readable programmatically with Selenium.

szoom = "0.25"
ch.ExecuteScript ("document.body.style.zoom = '" & szoom & "'")