I am a reseacher who has access to Twitter's new academic product track. I am experimenting with the Python wrapper searchtweets-v2 (found here) to try and call Tweets.
When I have previously used the API through an account with Premium access, tweets were returned with a large amount a metadata. However, when I try to use the below script, based on code from the searchtweets-v2 webpage, it only returns a tweet ID and text:
Code:
# Import packages
from searchtweets import ResultStream, gen_request_parameters, load_credentials, collect_results
# Load in previously saved credentials
search_args = load_credentials("~/.twitter_keys.yaml",
yaml_key="search_tweets_v2",
env_overwrite=False)
# Query inputs
search_terms = "snow"
add_terms = ' lang:en place_country:GB'
max_results = 100
start_time = '\"2020-02-01T00:00:00Z\"'
end_time = '\"2020-02-08T00:00:00Z\"'
# Build queries
tweet_full_query = '{"query":' + '"' + search_terms.replace('"', '\\"') + add_terms.replace('"', '\\"') + '"' + ',"max_results":' + str(max_results) + \
',"start_time":' + str(start_time) + ',"end_time":' + str(end_time) + '}'
# Call Tweets using 'collect_results'
tweets = collect_results(tweet_full_query,
max_tweets=100,
result_stream_args=academic_search_args)
# View results
tweets
Output (first 6 tweets only)
[{'data': [{'id': '1225926868592885765',
'text': '@JoeSalter89 I have, I recall a lot of meat being attacked with swords. Snow would’ve said it was an excellent victory it’s just a shame Wellington was Irish and his wife was a drug-addled cricket. Casual xenophobia and misogyny are his hallmark . I’m quoting his exact words as I imagine them'},
{'id': '1225926288583643141',
'text': '@ElfynEvans @TGR_WRC @CoedyBreninFP @RallySweden @TweeksCycles @bikeonscott Best of luck on the snow @ElfynEvans ! 👍 Now you just need to maintain @TGR_WRC ‘s winning record at @RallySweden 😁👍....no pressure eh! 😉😁'},
{'id': '1225926214608728065',
'text': '@MissElks What the fuck bro why so much snow'},
{'id': '1225926091690446849',
'text': 'Full Snow Moon in Leo on February 8/9 at 20 degrees will bring forth everything that lies in our heart center when it comes to our creativity, our love lives, and our ego. We will be thinking long and hard about our… https://url'},
{'id': '1225924864147677186',
'text': '@JoeSalter89 Snow is a bully. He recently disinterred the bones of Nelson to ask him why the hell he dragged Villeneuve across the Atlantic and back instead of fucking him over in Brittany. He’s drunk on power. And meths.'},
{'id': '1225922095168839680',
'text': '@leftmidfielder @Jay29ers @museumofjerseys It is true. Your atmospheric short story on the disused rail line in the snow hinted at a lot more if you set your mind to it.'}...
I have also tried using the ResultStream function, but get an identical output:
# Call tweets using 'ResultStream'
rs = ResultStream(request_parameters=tweet_full_query,
max_results=500,
max_pages=1,
**academic_search_args)
tweets = list(rs.stream())
# View results
tweets
I have tried exploring these objects, but they don't appear to have any additional metadata that I can find.
Does anyone have any suggestions on how to call historical Tweets with metadata using the new Academic product track? Ideally this would be through searchtweets-v2. However, other suggestions are also welcome!
I found a workaround or sorts. I did't realise that you now have to add additional query parameters to recieve more that basic information about each tweet. These query parameters are summarised here.
I didn't manage to adapt searchtweets-v2 to this task. However, I found Andrew Edward's code, published here, worked very well.