Search code examples
pythonapitwitterwebapitwitterapi-python

Python: TwitterRequestError with v2 full archive search


I'd like to get Tweets using the v2 full archive search. I don't understand the error I got from the code below. Did I request too many times?

Here's config.py

from TwitterAPI import TwitterAPI, TwitterPager
import csv


SEARCH_TERM = '#metoomen lang:en'
PRODUCT = 'fullarchive'
LABEL = 'prod'

api = TwitterAPI(api_key, 
             api_secret_key, 
             access_token, 
            access_token_secret)

r = TwitterPager(api, 'tweets/search/%s/:%s' % (PRODUCT, LABEL),
        {'query':SEARCH_TERM, 
        'fromDate':'201710170000',
        'toDate':'201801312359',
        "maxResults":500
        }).get_iterator()
    

csvFile = open('data.csv', 'w',encoding='UTF-8')
csvWriter = csv.writer(csvFile)

for item in r:
    csvWriter.writerow([item['created_at'],
                    item["id_str"],
                    item["source"],                    
                    item['user']['screen_name'],
                    item["user"]["location"],
                    item["geo"],
                    item["coordinates"], 
                    item['text'] if 'text' in item else item])

Here's the error I got.


TwitterRequestError: ('{"error":{"message":"Request exceeds account’s current package request limits. Please upgrade your package and retry or contact Twitter about enterprise access.","sent":"2021-06-09T09:54:54+00:00","transactionId":"8f5af84751ad0d30"}}',) (429): {"error":{"message":"Request exceeds account’s current package request limits. Please upgrade your package and retry or contact Twitter about enterprise access.","sent":"2021-06-09T09:54:54+00:00","transactionId":"8f5af84751ad0d30"}}

Solution

  • This error indicates that you are using the premium v1.1 API and have made too many calls. You need to switch to using the v2 full archive search URL. Note that this requires you to have an account with academic access.

    Something like

    api = TwitterAPI(consumer_key, consumer_secret, access_token_key, access_token_secret, api_version='2')
    
    r = api.request('tweets/search/all', {
        'query':QUERY, 
        'tweet.fields':'author_id',
        'expansions':'author_id'})
    

    There are examples in the TwitterAPI GitHub repository.