There is a lot to find about this subject, but can't figure this out. I need to scroll to the end of the page of a (not so long) infinity scroll. I have 2 options that work with chrome non-headless but doesn't seem to work headless.
The first one which i liked the most, works beautiful and found here on SA:
driver = webdriver.Chrome('c:/cd.exe', chrome_options=chrome_options)
driver.get('http://www.website.com')
while True:
count = len(driver.find_elements_by_xpath('//div[@itemprop="itemListElement"]'))
print(count)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
WebDriverWait(driver, 50).until(EC.visibility_of_element_located((By.XPATH,
"//div[@itemprop='itemListElement'][%s]" % str(count + 1))))
except TimeoutException:
break
Second more hack job after realizing i can't get away with above in headless mode:
driver = webdriver.Chrome('c:/cd.exe', chrome_options=chrome_options)
driver.get('https://www.website.com')
while True:
count = len(driver.find_elements_by_xpath('//div[@itemprop="itemListElement"]'))
actions = ActionChains(driver)
actions.send_keys(Keys.PAGE_DOWN)
actions.perform()
actions.send_keys(Keys.PAGE_DOWN)
actions.perform()
# focus_element_scroll = driver.find_elements_by_xpath('//section[@class="occasion-content"]')
# driver.find_elements_by_xpath('//div[@itemprop="itemListElement"]')[-1].send_keys(Keys.PAGE_DOWN)
# driver.find_elements_by_xpath('//div[@itemprop="itemListElement"]')[-1].send_keys(Keys.PAGE_DOWN)
# self.driver.find_element_by_css_selector("ul.list-with-results").send_keys(Keys.ARROW_DOWN)
print(count)
# driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
WebDriverWait(driver, 50).until(EC.visibility_of_element_located((By.XPATH,
"//div[@itemprop='itemListElement'][%s]" % str(count + 1))))
except TimeoutException:
break
So both work in chrome but don't in headless mode, i need to push them to a ubuntu vps where they need to be headless, i know of the xvfb option but i am glad i could remove that and use native chrome since the droplets don't have much memmory.
Edit: Just tried this approach with focus on an element in the footer, also works in non-headless but doesn't in headless:
ActionChains(driver).move_to_element(focus[0]).perform()
Someone got a different approach?
Edit Just want to know if it is possible to scroll with chrome in headless mode!
Found the answer after 2 days trying different combinations of versions of selenium, chrome and chromedriver i alsmost gave up and wanted to go with the xvfb.
Already tried to maximze the window in the chrome arguments, that didn't help. But this time i tried setting a manual window size. That helped.
chrome_options.add_argument("window-size=1920,1080")
Posting here so that the next one wont take as long as me.