So I've been working on a project recently, and I need to get a user's followers list.
To do so, I use Selenium in Python.
I first log in, and after clicking a few buttons, I click on the Followers button of the user.
Then, a "box" appears, with some of his followers.
However, the list only contains 12 followers, and to load more, you need to scroll all the way down, wait a bit, then scroll again etc...
So my problem is I don't know how to scroll through this box! I looked on the website searching for some solutions to my problem, but all of them didn't work for me. One of the possibilities is that maybe I selected the wrong element, but I tested a lot of XPath and still none worked.
So if someone could help me I would be very grateful!
So here's my code :
# After clicking on his followers button on his profile button
scrollable_list = driver.find_element(By.XPATH, "/html/body/div\[2\]/div/div/div\[3\]/div/div/div\[1\]/div/div\[2\]/div/div/div/div/div\[2\]/div/div/div\[3\]"
)
# Scroll 3 times
for i in range(3):
driver.execute_script("arguments\[0\].scrollTop = arguments\[0\].scrollHeight", scrollable_list)
# Wait some time for the rest of the list to load
time.sleep(1.5)
# Get the usernames of the list
Followers_elements = driver.find_elements(By.XPATH, "/html/body/div\[2\]/div/div/div\[3\]/div/div/div\[1\]/div/div\[2\]/div/div/div/div/div\[2\]/div/div/div\[3\]/div\[1\]/div"
)
Followers_elements = \[element.text for element in Followers_elements\]
Sorry if my code is a bit messy, I'm kind of new to code, so besides my problem, if you have some bits of advice for me to write better code, let me know And I'm sorry if you struggle to read my text, I'm not an English speaker.
It looks like you're escaping things wrong. Try:
driver.execute_script("""
arguments[0].scrollTop = arguments[0].scrollHeight
""", scrollable_list)
if you're not sure you're scrolling the right element, scroll it down a bit and do this in js console.
el = $$('*').find(el => el.scrollTop)