Transferring Twitter Tweets to a txt file

Question

from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import json 
from pprint import pprint

data_file = open('twitter.json')  
data = json.load(data_file)
##Json file with all the ckey, csecret, atoken, and asecret
pprint(data)

#consumer key, consumer secret, access token, access secret.
ckey = data["ckey"]
csecret = data["csecret"]
atoken = data["atoken"]
asecret = data["asecret"]

class listener(StreamListener):

def on_data(self, data):
    all_data = json.loads(data)       
    tweet = all_data["text"]        
    username = all_data["user"]["screen_name"]
    print((username,tweet))
    return True

def on_error(self, status):
    print (status)


auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)

The code above is all standard in accessing the twitter api. However, I need to transfer the tweets obtained from twitter to a .txt file. I tried using the code below twitterStream = Stream(auth, listener())

fid = open("cats based tweets.txt","w")
for tweet in twitterStream.filter(track=[cats]):
        fid.write(tweet)
    fid.close()

I intend on finding all twitter tweets/reposts that include the keyword cats, which it does. However, it is supposed to also write a txt file that includes all the tweets but it doesn't. Can anyone tell me what I need to do it fix it.

EDIT : I used the code that you guys have written but it doesn't return all of the tweets. It prints out like 5 or 6 then the error

RuntimeError: No active exception to reraise

appears and I have no idea why. Why does this occur cause I know it shouldn't.

Answer 1

I've done this in a project and my method involves changing the on_data method within the StreamListener object. My code looks like this:

class Listener(StreamListener):
    def __init__(self, api=None, path=None):
        #I don't remember exactly why I defined this.
        self.api = api
        #We'll need this later.
        self.path = path

    def on_data(self, data):
        all_data = json.loads(data)

        tweet = all_data["text"]        
        username = all_data["user"]["screen_name"]
        print((username,tweet))

        #Open, write and close your file.
        savefile = open(file_path, 'ab')
        savefile.write(tweet)
        savefile.close()

        return True

A few things in the actual code, not where you redefined Listener or on_data. In order:

1. Define the file where you want to save. Let's call that variable the file_path. Don't forget to add the .txt extensions here.

2. Call the Stream and the Listener:

   twitterStream = Stream(authorization, Listener(path=file_path))
   

3. Use your filters. Mine are coordinates and I put the filter in a try, except so that my code doesn't stop. Here it is adapted for you:

   try:
       twitterStream.filter(track=[cats])
   except Exception, e:
       print 'Failed filter() with this error:', str(e)
   

Now the text in the tweet should be written in the file whenever a text appears in the stream. Take a look at your file size and you should see it increase. Particularly, if your filter is about cats. Internet loves cats.