Search code examples
pythonpandastwittergeolocationlatitude-longitude

Parse User name for extracting user location Twitter


I am trying to scrape user location with respect to user names from twitter.

Input: The user list has more than 50K User names

AkkiPritam,6.77E+17,12/15/2015,#chennaifloods
AkkiPritam,6.77E+17,12/15/2015,#bhoomikatrust
AkkiPritam,6.77E+17,12/15/2015,#akshaykumar
gischethans,6.77E+17,12/15/2015,#chennaifloods
mid_day,6.77E+17,12/15/2015,#bollywood
mid_day,6.77E+17,12/15/2015,#chennaifloods
Nanthivarman16,6.77E+17,12/15/2015,#admkfails
Nanthivarman16,6.77E+17,12/15/2015,#jayafails
Nanthivarman16,6.77E+17,12/15/2015,#stickergovt
Nanthivarman16,6.77E+17,12/15/2015,#chennaifloods
AdilaMatra,6.77E+17,12/15/2015,#chennaifloods
AdilaMatra,6.77E+17,12/15/2015,#climatechange
AdilaMatra,6.77E+17,12/15/2015,#delhichokes
AdilaMatra,6.77E+17,12/15/2015,#smog
HDFCERGOGIC,6.77E+17,12/15/2015,#chennaifloods
HDFCERGOGIC,6.77E+17,12/15/2015,#tnfloods
ImSoorej,6.77E+17,12/15/2015,#chennaifloods
ImSoorej,6.77E+17,12/15/2015,#chennaimicr

Code: I want to find geo location possibly geo coordinates.

from __future__ import print_function
import tweepy
from tweepy import OAuthHandler
from tweepy import Stream
from tweepy.streaming import StreamListener
import pandas as pd
import csv

consumer_key = 'xyz'
consumer_secret = 'xyz'
access_token = 'xyz'
access_token_secret = 'xyz'

data = pd.read_csv('user_keyword.csv')
df = ['user_name', 'user_id', 'date', 'keyword']

def get_user_details(username):
        userobj = api.get_user(username)
        return userobj

if __name__ == '__main__':
    #authenticating the app (https://apps.twitter.com/)
    auth = tweepy.auth.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)
    api = tweepy.API(auth)

    username = df['user_name']
    userOBJ = get_user_details(username)
    print(userOBJ.location)

Error: Trouble parsing the usernames into program.

Traceback (most recent call last):
  File "user_profile_location.py", line 38, in <module>
    username = df['user_name']
TypeError: list indices must be integers, not str

Solution

  • You are using 'data' to define your DataFrame and 'df' for what I think should be the columns of the DataFrame

    data = pd.read_csv('user_keyword.csv')
    df = ['user_name', 'user_id', 'date', 'keyword']
    

    I assume that the user_keyword.csv file has no header, try adding:

    data.columns = df
    

    It will change the column names to the values stored in df. Then later instead of:

    username = df['user_name']
    

    Try:

    username = data['user_name']
    

    Keep in mind that now username is a whole column so get_user_details(username) should not be expecting a single string.