I am building a module source_collect.py that shall contain modules I will use to to collect data from various sources and return raw data that I can use a second module to parse.
I am able to print or write the streaming data to a json file, but I am not able to return the data through the return function. Do anybody have som pointers on what i am missing out on?
import tweepy
import twitter_credentials as cred
class my_streaming_object(tweepy.streaming.StreamListener):
def on_data(self,raw_data):
try:
print('......streaming...')
self.raw_data = raw_data
return self.raw_data
except BaseException as e:
print(f'Code broke :{str(k)}')
return False
def on_error(self,status_code):
if status_code ==420:
print(f'Twitter limit stop{self.status.code}')
return False
class twitter_data:
def stream_tweets_keywords(self, keywords):
# get twitter access
authen = tweepy.auth.OAuthHandler(cred.CONSUMER_KEY,cred.CONSUMER_SECRET)
authen.set_access_token(cred.ACCESS_TOKEN,cred.ACCESS_TOKEN_SECRET)
test = authen.get_username()
print(f'You are Authenticated as Twitter user {test}')
print('....Initialising twitter stream......')
# stream tweets
try:
streamer = my_streaming_object()
my_stream = tweepy.streaming.Stream(authen, listener=streamer)
my_stream.filter(track=keywords, is_async=True)
return
except BaseException as b:
print(b)
return False
if __name__=='__main__':
run = twitter_data()
run.stream_tweets_keywords(['Hydrogen','Nikola'])
Solved it by changing my logic. Made a function in the DB_store module that I used within this module handling the streaming response to db.
The change in logic was instead of having this module return then picking it up in the next function input. I nested the next function in this one.
I dont now if this is a good practice, so if any body have a good OOP guideline for the overall logic it would be appreciated.