Search code examples
pythonseleniumtwitter

Scraping followers from Twitter using Selenium

I'm new to Python, and have been trying to code an app that can scrape data from Twitter. I tried searching all similar possible solutions on stack, and internet but failed.

I want to scrape all these usernames: See here

This is my code:

driver.get("https://twitter.com/twitterusername/followers")
sleep(10)

usernames = driver.find_elements_by_class_name("css-901oao.css-16my406.r-poiln3.rbcqeeo.r-qvutc0")
for username in usernames:
    print(username.get_attribute("href"))

The result I get:

None
None
None
None
None
None
None

... continued

Thank you for your help.

Solution

  • so, using BeautifulSoup is off the table. we can use only selenium to handle this.

    for a in driver.find_elements_by_xpath('//div[@aria-label="Timeline: Followers"]//a[@role="link"]'):
    url = a.get_property('href')
    if 'search' in url:
        return 
    print(url.replace("https://twitter.com/", "@")