Search code examples
pythontwittergeolocationgeojsondata-collection

Python API Data Collection Social Media API and Geolocation I.E. Twitter

  • Can Anyone Help me out here with working through some logic with Python3 for Data collection or propose any easier methods that are as efficient. I know you can add data sources to programs like Tableau and others but that's not really what i'm doing unless you're certain it has that functionality easily built in.

How would I go about my logic for a data collection program written in python to collection a series of tweets all using the say hashtag, #sample, and then I get their location from where --- GeoJson? I'm assuming..

For those of you more experience with data collection with python3 can I add in logic that will allow me to collect using a twitter api app or py tweety for example. I just am wondering can I have every 10th collected one trigger a secondary action that wont tax the performance of a fast moving stream.

Use-Case Scenario: For instance like a Stream or Data Collection of all the tweets from a Kenny Chesney concert, which I know is 1mi square between long1 lat1 and long2 lat2 making a square around the arena exenting as far out as a mile I want to stream all the tweets about the olympics the hard way, I way to collect it not fetch feeds. Can someone give me the one-two on collecting with python and twitter and how well geolocation works for those users whose posts are geolocation=on. For every person with the same geolocation on the same datetime +- 1 hour tweeting with a #samplekenny hashtag I want to continue to collect and alert myself or add names to txt file or some trigger. (realizing there are many things that I realize are conditional/dependent based on a number of factors but lets says the worlds perfectly flat and the arena itself is a square mile. As close as anyone can get, i'm mostly concerned about specifying my geolocation needs and triggering some sort of action to inform, like a javaascript trigger to email/sms/.alert() .snack() .onClick(doSomething); )

Anything close to this, even if it's pieced together a bit like the question, from different individuals I'd be grateful, i've done this before, not with python, i've gotten to really enjoy python but I cant say that I KNOW it yet.

I think i'm close, i'm wanting to find the least taxing way to do this, lets say for events, and then if there is an overall less taxing way, otherthan limiting your collection group with stricter filters and such. only would need text and encoding symbols. and tweetId

Solution

  • You could use the command-line scripts that come with this python package which does almost exactly what you are looking for: https://github.com/geduldig/TwitterGeoPics

    You can use the REST API:

    python -u -m TwitterGeoPics.SearchOldTweets -words #sample -location nyc

    Or, you can use the Streaming API this way:

    python -u -m TwitterGeoPics.StreamNewTweets -words #sample -location nyc

    These scripts lookup the latitude and longitude for a place. If you want to specify the latitude and longitude yourself, simply strip away the lookup code.

    Most people do not turn geo location, so don't expect many results.