How to scroll down locations on google maps

Question

I am trying to load 20 enginnering schools from google maps using the code below. The problem is that I need to scroll down the results in order to load more results, otherwise I get only 8 results.

Can someone help please;

Many thanks in advance

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time

chrome_options = Options()
chrome_options.add_experimental_option("detach", True)

driver = webdriver.Chrome(options=chrome_options)

driver.get("https://www.google.com/")
driver.find_element(By.ID, "L2AGLb").click()

driver.get("https://www.google.com/maps")

# Wait for the page to load and display the search box
time.sleep(3)

# Input the search query for "patisserie in London" and press Enter
search_box = driver.find_element(By.ID, "searchboxinput")
search_box.send_keys("engineering school in london")
search_box.send_keys(Keys.RETURN)

# Wait for the search results to load
time.sleep(5)

# Get the elements based on the below XPath locator
results = driver.find_elements(By.XPATH, "//a[@class='hfpxzc']")

# Print the names for debugging
for i, result in enumerate(results[:20], start=1):
    print(f"Result {i}: {result.get_attribute('aria-label')}")

# Close the browser
driver.quit()

I am trying to load 20 enginnering schools from google maps using the code below. The problem is that I need to scroll down the results in order to load more results, otherwise I get only 8 results.

Answer 1

You have 4 problems there:

1. Map items tab is not focused by default.
2. New items are loaded async, so after scroll you need to wait for condition that they are rendered.
3. Scrolling to the last item not triggering re-render, you should scroll a bit down.
4. Scroll via Selenium ActionChains resets focus.

My solution is dirty, but you can start from it and improve if you want:

1. Init while loop
2. Get array of map items
3. Scroll to the last item via ActionChains
4. Focus tab by clicking on element on in the left bottom point of input area
5. Execute with small delay ARROW_DOWN events
6. Wait for last element position to be stable
7. Get new array of map items
8. If last element is previous map item element - break loop, else - repeat.

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains, Keys

actionChains = ActionChains(driver)
wait = WebDriverWait(driver, 20)

def wait_for_element_location_to_be_stable(element):
    initial_location = element.location
    previous_location = initial_location
    start_time = time.time()
    while time.time() - start_time < 1:
        current_location = element.location
        if current_location != previous_location:
            previous_location = current_location
            start_time = time.time()
        time.sleep(0.4)

# your previous code

results = driver.find_elements(By.XPATH, "//a[@class='hfpxzc']")

break_condition = False
focus_element = driver.find_element(By.ID, 'zero-input')
while not break_condition:
    temp = results[-1]
    actionChains.scroll_to_element(results[-1]).perform()
    actionChains.move_to_element(focus_element).click().perform()
    for i in range(3):
        actionChains.send_keys(Keys.ARROW_DOWN).perform()
        time.sleep(0.5)
    wait_for_element_location_to_be_stable(temp)

    results = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//a[@class='hfpxzc']")))
    if results[-1] == temp:
        break_condition = True

for i, result in enumerate(results[:20], start=1):
    print(f"Result {i}: {result.get_attribute('aria-label')}")