im currently working on a scraper for public goole sheets and noticed that I have to fully load the spreadsheet before importing it into bs4. For that I wanted to use selenium to open the page and scroll to the bottom but its harder than I thought.
My current aproach was using:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
but its doing nothing. The only way I could move the page was by action chains.
So if anyone has a idea how to fix this or how the hole selenium part can be skipped(by usng something that loads the complete page at once) let me know.
Don't try to use scrollTo. Instead just send the keyboard command that will get you to the last row instead. Ctrl+Down arrow or Cmd+Down arrow is the keyboard shortcut.
In JavaScript it would be something like
await driver.actions()
.sendKeys(Key.CONTROL) // Or Key.COMMAND for Mac
.sendKeys(Key.DOWN)
.perform()