Search code examples
pythontwitterstreamingdevicetweepy

Tweepy's Streaming API does not recognize tweets from different devices


This is a really odd and specific question. I am developing a twitter bot that users can tweet quotes to, and the bot will in turn take those tweets and develop an inspirational picture to go along with their quote. So for example, let's say I'd tweet: @fake_quotes_bot "I'm gonna starve myself until they listen" - Ghandi. Well then it'd take that quote and that person next to the hyphen and generate an image.

Moving on from the generalities, I had just programmed a quoting filter to make sure the bot can grab quotes in the most efficient way. So, for example, this would not be of use: @fake_quotes_bot "Hello'this is" ' "a " quote" - person. Within this quoting filter, if a user misquotes their tweet (as seen), my bot will autoreply instructions on how to correctly structure their tweet. And upon running the bot in PyCharm on my desktop and then tweeting at the bot with a different account, all works great. The error messages are met, and if a tweet is correctly constructed, it will approve the tweet. The problem arises, however, when I then send a tweet from a different device other than the desktop computer that the bot is running on. The logic that seemingly worked perfectly when tweeting from a desktop has now fallen flat on its face when receiving a tweet through, say, an iPhone. No matter what tweet I throw out there, the same error message is received by the bot.

Here's my code:

import tweepy
import json

consumer_key, consumer_secret = ###, ###
access_token, access_token_secret = ###, ###

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)


def Data_Analysis(tweet, tweet_data):
    def Data_Write():
        print("DATA FOR THIS TWEET:", tweet_data, "\n")

    def Quote_Filter():

        print("INCOMING TWEET: " + "  > " + str(tweet) + " <   " + " FROM: @" +
              str(tweet_data.get('user', '').get('screen_name', '')) + "/" + tweet_data.get('user', '').get('name', ''))

        def Profanity_Filter():
            pass

        def Person_Filter():
            #WIP for now
            print("Filtering people...", end=" ")
            print("SUCCESSFUL")
            print("APPROVED TWEET: " + tweet)
            print("APPROVED TWEET DATA:", tweet_data, "\n")

        def Quotation_Marks_Filter():

            print("Filtering quotation marks...", end=" ")

            # Filters out tweets that contain quotes
            if '"' in tweet or "'" in tweet:
                double_quote_count = tweet.count('"')
                single_quote_count = tweet.count("'")

                # Double Quotes quote
                if double_quote_count > 0 and single_quote_count == 0:
                    if double_quote_count > 2:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: Please refrain from using too many quotation marks.",
                                          tweet_data.get('id'))

                        print("ERROR: Please refrain from using too many quotation marks. \n")
                    elif double_quote_count == 1:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: Only a singular quote was entered.",
                                          tweet_data.get('id'))

                        print("ERROR: Only a singular quote was entered. \n")
                    # Pass through to other filter
                    else:
                        print("SUCCESSFUL")
                        Person_Filter()

                # Single quotes quote
                elif double_quote_count == 0 and single_quote_count > 0:
                    if single_quote_count > 2:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: Please refrain from using too many quotation marks.",
                                          tweet_data.get('id'))

                        print("ERROR: Please refrain from using too many quotation marks. \n")
                    elif single_quote_count == 1:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: Only a singular quote was entered.",
                                          tweet_data.get('id'))

                        print("ERROR: Only a singular quote was entered. \n")
                    # Pass through to other filter
                    else:
                        print("SUCCESSFUL")
                        Person_Filter()

                # If a quote has two types of quotes
                else:
                    # Filter if there are too many quotes per character
                    if double_quote_count > 2:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: If you are implementing a quote within a quote or are abbreviating,"
                                          "please refrain from using more than two instances of a double quote."
                                          , tweet_data.get('id'))

                        print("ERROR: If you are implementing a quote within a quote or are abbreviating,"
                              "please refrain from using more than two instances of a double quote. \n")
                    elif double_quote_count == 1:
                        api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                          " ERROR: Could not identify the quote. If you are implementing a quote "
                                          "within a quote or are abbreviating,  please use two instances of the "
                                          "double quote.",
                                          tweet_data.get('id'))

                        print("ERROR: Could not identify the quote. If you are implementing a quote "
                              "within a quote or are abbreviating,  please use two instances of the "
                              "double quote. \n")

                    # If it's correct in its number, then figure out its beginning and ending quotes to pull text
                    else:
                        quote_indexes = []
                        quote_chars = []

                        indices = [index for index, value in enumerate(tweet) if value == '"']
                        for i in indices:
                            quote_indexes.append(i)
                            quote_chars.append('"')

                        indices = [index for index, value in enumerate(tweet) if value == "'"]
                        for i in indices:
                            quote_indexes.append(i)
                            quote_chars.append("'")

                        beginning_quote = quote_indexes.index(min(quote_indexes))
                        ending_quote = quote_indexes.index(max(quote_indexes))

                        # If the starting and ending quotes are similar (I.E. " and ") then pass through to other filter
                        if quote_chars[beginning_quote] == quote_chars[ending_quote]:
                            print("SUCCESSFUL")
                            Person_Filter()

                        # Do not align
                        else:
                            api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                              " ERROR: The beginning and endings quotes do not align.",
                                              tweet_data.get('id'))

                            print("ERROR: The beginning and endings quotes do not align. \n")

            # No quote found
            elif '"' or "'" not in tweet:
                grab_user = tweet_data.get('user', '').get('screen_name', '')

                if grab_user == "fake_quotes_bot":
                    # If I were to test this on my own twitter handle, it would get stuck in an auto-reply loop.
                    # Which will probably ban me.
                    print("PASSING UNDER MY OWN SCREEN NAME... \n")

                if grab_user != "fake_quotes_bot":
                    api.update_status("@" + str(tweet_data.get('user', '').get('screen_name', '')) +
                                      " ERROR: This tweet does not contain a quote. Be sure to use quotation marks.",
                                      tweet_data.get('id'))

                    print("ERROR: This tweet does not contain a quote. Be sure to use quotation marks. \n")

        def Retweet_Filter():

            print("Filtering retweets...", end=" ")

            # Filters out tweets that are retweets
            if "RT" in tweet[0:3]:
                print("RETWEET. SKIPPING... \n")
            else:
                print("SUCCESSFUL")
                Quotation_Marks_Filter()

        Retweet_Filter()

    Quote_Filter()


class StreamListener(tweepy.StreamListener):

    def on_data(self, data):
        tweet_data = json.loads(data)
        if "extended_tweet" in tweet_data:
            tweet = tweet_data['extended_tweet']['full_text']
            Data_Analysis(tweet, tweet_data)

        else:
            try:
                tweet = tweet_data['text']
                Data_Analysis(tweet, tweet_data)
            except KeyError:
                print("ERROR: Failed to retrieve tweet. \n")


print("BOT IS NOW RUNNING. SEARCHING FOR TWEETS...\n")

Listener = StreamListener()
Stream = tweepy.Stream(auth=api.auth, listener=Listener, tweet_mode='extended')
Stream.filter(track=['@fake_quotes_bot'])

Output from tweeting from the same desktop:

INCOMING TWEET:   > @fake_quotes_bot "hello, stackoverflow!" <    FROM: @bulletinaction/BulletInAction
Filtering retweets... SUCCESSFUL
Filtering quotation marks... SUCCESSFUL
Filtering people... SUCCESSFUL
APPROVED TWEET: @fake_quotes_bot "hello, stackoverflow!"
APPROVED TWEET DATA: {###data###} 

Output if I were to send a tweet through my phone:

INCOMING TWEET:   > @fake_quotes_bot “heyyo, stackoverflow” <    FROM: @bulletinaction/BulletInAction
Filtering retweets... SUCCESSFUL
Filtering quotation marks... ERROR: This tweet does not contain a quote. Be sure to use quotation marks. 

Here's a youtube video of the code running, as I'm sure this is a very odd question: https://www.youtube.com/watch?v=skErnva4ePc&feature=youtu.be


Solution

  • Your phone is using left en right double quotation mark instead of quotation mark :

    "   U+0022 QUOTATION MARK
    “   U+201C LEFT DOUBLE QUOTATION MARK
    ”   U+201D RIGHT DOUBLE QUOTATION MARK
    

    (from : Are there different types of double quotes in utf-8 (PHP, str_replace)?)

    So just perform a replacement on the tweet text before your tests :

    tweet = tweet_data['extended_tweet']['full_text'] # as you did, then :
    tweet = tweet.replaceAll("[\\u2018\\u2019]", "'")
    tweet = tweet.replaceAll("[\\u201C\\u201D]", "\"");
    

    (from : Converting MS word quotes and apostrophes)