Search code examples
pythonpython-2.7twittertweepyapi-design

Tweepy - Exclude Retweets


Ultimate goal is to use the tweepy api search to focus on topics (i.e docker) and to EXCLUDE retweets. I have looked at other threads that mention excluding retweets but they were completely applicable. I have tried to incorporate what I've learned into the code below but I believe the "if not" piece of code is in the wrong place. Any help is greatly appreciated.

#!/usr/bin/python
import tweepy
import csv #Import csv
import os

# Consumer keys and access tokens, used for OAuth
consumer_key = 'MINE'
consumer_secret = 'MINE'
access_token = 'MINE'
access_token_secret = 'MINE'

# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)


api = tweepy.API(auth)
# Open/Create a file to append data
csvFile = open('docker1.csv', 'a')
#Use csv Writer
csvWriter = csv.writer(csvFile)


ids = set()
for tweet in tweepy.Cursor(api.search, 
                    q="docker", 
                    Since="2016-08-09", 
                    #until="2014-02-15", 
                    lang="en").items(5000000):
if not tweet['retweeted'] and 'RT @' not in tweet['text']:
    #Write a row to the csv file/ I use encode utf-8
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8'), tweet.favorite_count, tweet.retweet_count, tweet.id, tweet.user.screen_name])
    #print "...%s tweets downloaded so far" % (len(tweet.id))
    ids.add(tweet.id) # add new id
    print ("number of unique ids seen so far: {}",format(len(ids)))
csvFile.close()

Error Message


Solution

  • Filtering at API level:

    q='your_search -filter:retweets'

    read more on this here.

    Dumb way is to filter in code

    So tweet is an object not a JSON or dict, you should not access it like tweet['retweeted'] and tweet['text']

    Instead use this line :

    if not tweet.retweeted:
    

    Or for your use case :

    if (not tweet.retweeted) and ('RT @' not in tweet.text):