Search code examples
pythonpython-requeststwitter-rest-api

How to perform oauth when doing twitter scraping with python requests


I am trying to retrieve 100 recent tweets of a user. It is working well with tweepy module in Python. But how can I do the same with requests in python. I want to do:

import requests
r = requests.get('https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=xxxx&count=100')

Here how to do the authentication with client key, client secret, access token and access secret, before sending this request?


Solution

  • You can use requests-oauthlib as described in requests docs.

    OAuth:

    import requests
    from requests_oauthlib import OAuth1
    url = 'https://api.twitter.com/1.1/account/verify_credentials.json'
    auth = OAuth1(API_KEY, API_SECRET, ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
    requests.get(url, auth=auth)
    

    Get tweets:

    r = requests.get('https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=stackoverflow&count=100', auth=auth)
    for tweet in r.json():
      print tweet['text']