Search code examples
jsonpandastwittermining

Twitter TypeError: 'int' object has no attribute '__getitem__'

I am trying to follow a tutorial on twitter data mining, the steps emulated as as follows:

 tweets_data_path = '/home/ambijat/ipythonnbs/twitter/twitter_data.txt'
    tweet_data = []
 tweets_file = open(tweets_data_path, "r")
for line in tweets_file:
try:
    tweet = json.loads(line)
    tweet_data.append(tweet)
except:
    continue

And then:

   tweets = pd.DataFrame()
  tweets['text'] = map(lambda tweet: tweet['text'], tweet_data)
   tweets['lang'] = map(lambda tweet: tweet['lang'], tweet_data)
  tweets['country'] = map(lambda tweet: tweet['place']['country'] if tweet['place'] != None else None, tweet_data)

And the outcome is:

    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
  <ipython-input-9-a42fce63cc05> in <module>()
     1 tweets = pd.DataFrame()
----> 2 tweets['text'] = map(lambda tweet: tweet['text'], tweet_data)
  3 tweets['lang'] = map(lambda tweet: tweet['lang'], tweet_data)
     4 tweets['country'] = map(lambda tweet: tweet['place']['country'] if tweet['place'] != None else None, tweet_data)

   <ipython-input-9-a42fce63cc05> in <lambda>(tweet)
  1 tweets = pd.DataFrame()
   ----> 2 tweets['text'] = map(lambda tweet: tweet['text'], tweet_data)
  3 tweets['lang'] = map(lambda tweet: tweet['lang'], tweet_data)
  4 tweets['country'] = map(lambda tweet: tweet['place']['country'] if tweet['place'] != None else None, tweet_data)

   TypeError: 'int' object has no attribute '__getitem__'

Could someone please help me in locating my mistake I am practically novice.

Solution

  • You can also directly pass tweet_data list to json_normalize:

    from pandas.io.json import json_normalize    
tweets = json_normalize(tweet_data)[["text", "lang", "place.country"]]

    text                                                lang    place.country
0   This not the 1st. They hv 1 in Faisalabad alre...   en      پاکستان
1   RT @TOLOnews: Pakistan Trying To Create Third ...   en      NaN
2   RT @murtazasolangi: JuD establishes parallel "...   en      NaN