My challenge is to fetch all the list of followers of an account with over 30 million followers.
Currently i'm using the GET followers/list endpoint on Twitter's REST API, however, with the rate limiting of the free API this takes lots of days to achieve.
I am willing to pay Twitter for Premium API access, however I could not find any data suggesting that the Premium API has the necessary endpoints and sufficient rate limiting to help resolve this task in a matter of hours at the most.
Would appreciate any insight on this matter...
You can get 1 000 000 followers (or friends) within about 6 hours with public API like this :
1st step: get followers or friends ids with GET followers/ids or GET friends/ids. The rate limit is 15 requests in 15 mn to get 5000 user id per request. This is 1 request each minute then 200 mn to get 1,000,000 user ids (3h20mn)
2nd step: convert user ids to user names (screen names) with GET users/lookup. The rate limit is 900 requests within 15 mn to convert 100 user ids per request. So 100 users per second, this is 1,000,000 users in 10,000 s (2h46mn)
Total : 6h06 to get 1,000,000 screen names of followers (or friends)
With premium API it seems there is no rate limiting so it should be faster (https://developer.twitter.com/en/docs/basics/rate-limiting)...