Scrape Data from Twitter using Snscrape, return error with Tweet object has no attribute 'likeCount'

Question

Follow the code

import snscrape.modules.twitter as sntwitter
import pandas as pd

# Creating list to append tweet data to
attributes_container = []

# Using TwitterSearchScraper to scrape data and append tweets to list
for i,tweet in enumerate(sntwitter.TwitterSearchScraper('sex for grades since:2021-07-05 until:2022-07-06').get_items()):
    if i>150:
        break
    attributes_container.append([tweet.user.username, tweet.date, tweet.likeCount, tweet.sourceLabel, tweet.content])
    
# Creating a dataframe to load the list
tweets_df = pd.DataFrame(attributes_container, columns=["User", "Date Created", "Number of Likes", "Source of Tweet", "Tweet"])

but it returned the error with: AttributeError: 'Tweet' object has no attribute 'likeCount'.

Did anyone know why it happen?

Thanks.

Answer 1

You need to install the developer version of snscrape to get attributes like likeCount. You can either try to install the developer version and run your code again; or if you're ok with only what your current version gives you, you can try:

for i,tweet in enumerate(sntwitter.TwitterSearchScraper('sex for grades since:2021-07-05 until:2022-07-06').get_items()):
    if i>150:
        break
    
    attributes_container.append(
        dict(
            (name, getattr(tweet, name)) for name in dir(tweet) 
            if not name.startswith('_') and name not in ['count','index']
        ) 
    )
    
tweets_df = pd.DataFrame(attributes_container)

and then tweets_df.columns gives me the output:

Index(['content', 'date', 'id', 'outlinks', 'outlinksss', 'tcooutlinks',
       'tcooutlinksss', 'url', 'username'],
      dtype='object')

and these are the only data I can access with my version of snscrape. [You might get something different.]