Search code examples
pythonjsonparsingtweets

No module named ijson

I have a big json file of tweets(around 5GB). I am having memory error. So, I decided to parse the data. I found ijson package. I am having such an error:

import ijson
parser = ijson.parse(tweets_data_path )
tweets_data = []
f = open(tweets_data_path, "r")
objects = ijson.items(f, 'other_config.item')
for line in objects:
    try:
        tweet = json.loads(line)
        tweets_data.append(tweet)
    except:
        continue

"No module named ijson" I am fairly new to Python, I looked at the source file of the package. But I could bout quite get what the requirements are. Any help will be appreciated.

Solution

  • ijson is an external package that is not included with the regular python libraries. You need to install ijson yourself first. Look into using something like pip which is a package manager for python. Once installed you can install ijson through the terminal like this:

    pip install ijson