My situation is the following: I am writing a bot to automate scrolling through instagrams explore page. I want to like the first 100 pictures that appear for a given hashtag.
I am using: Selenium, python, chrome.
My problem is the following: Whenever I scroll, new posts “appear” and old ones “disappear”. I am using an xpath like this: //article[row]. However, this only works for the first 10 rows since then row 11 is actually row 8 because previous pictures that I have already passed do no longer appear on the same level.
Does somehow know how to handle such a situation? I would like to access every single picture in a dynamically changing page where new pictures are added when I scroll down and older ones disappear.
Thank you so much!
By chance I found an answer to this question: Find next sibling element in Selenium, Python?
Basically you have to use JS: next_sibling = driver.execute_script("""return arguments[0].nextElementSibling """, element)