Unable to track multiple keywords with separate TwitterStream instances

Question

I am trying to run multiple TweetStream clients and track keywords, each with different access/oauth tokens (created on different Twitter accounts). I am using sidekiq to start each TweetStream::Client instance.

I would expect that because I should be able to start each sidekiq job and track each keyword respectively. However, what I am finding is that as soon as the firstTweetStream::Client is connected and tracking, any clients intialized after that point immediately are "processed" aka completed, in sidekiq and do not track the keywords.

def perform(app_id)    
  app = App.get(app_id)
  client = prepare_twitter_client(app)
  stream_tweets("google", app_id, client)
end

def prepare_twitter_client(app)
    # Use the companies twitter app token to connect to the stream
    @twitter_client = nil
    @twitter_client = TweetStream::Client.new
    @twitter_client.consumer_key               = app.tw_consumer_key
    @twitter_client.consumer_secret            = app.tw_consumer_secret
    @twitter_client.oauth_token                = app.tw_oauth_token
    @twitter_client.oauth_token_secret         = app.tw_oauth_secret
    @twitter_client.auth_method                = :oauth
    return @twitter_client
end


def stream_tweets(company_domain, app_id, twitter_client)
    twitter_client.on_error do |message|
    end.track(company_domain) do |status|
        logger.info "Tweet found that match the criteria.  #{status.text}"
        create_new_impression(status, app_id)
    end
end

I suspect TweetStream is having problems creating separate instances and tracking keywords from multiple clients..

Answer 1

Solved.

TweetStream does not appear to be threadsafe. Even though I would recreate a new TwitterStream::Client for every sidekiq job instance, and even had different Twitter accounts/apps, I was unable to run multiple jobs simultaneously.

I replaced TweetStream with the Sferik Twitter Gem and it worked perfectly.