R rtweet: search_tweets loop does not continue if no results returned for a given Twitter handle

Question

I have a dataframe of Twitter handles. When I loop over the handles using the search_tweets function, the loop stops collecting tweets if one of the Twitter handles does not return any results.

I would like to construct the loop such that if no results are returned, it ignores the handle and moves to the next.

My handle dataframe looks like this:

handles=data.frame(`Twitter Handle`=c("@_CHKD","@AIDHC","@BannerChildrens","@BaptistOnline"))

And the loop looks like this:

# Loop through the twitter handles & store the results as individual dataframes
for(handle in twitter_handles) {
  result <- search_tweets(handle, n = 3500 , include_rts = FALSE,retryonratelimit = TRUE)
  result$`Twitter Handle` <- handle
  result$Source <- "Search"

  df_name <- paste(tolower(substring(handle, 2)),"_search")

  if(exists(df_name)) {
    assign(df_name, unique(rbind(get(df_name), result)))
  } else {
    assign(df_name, result)
  }
}

When I run the loop, it throws the following error after it encounters a handle that returns nothing:

Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column

I have tried to search online for a solution but I've not been successful. Any pointers would be very helpful.

Answer 1

So for me, I do not see an error when I search_tweets for a handle with no tweets (i.e "@BannerChildrens"), instead I return an empty data.frame of length 0. By adding an if statement you can exclude all handles with no tweets. The following code returns three dataframes ("@_CHKD","@AIDHC","@BaptistOnline") that are in my global environment, with no errors.

handles=data.frame(`Twitter Handle`=c("@_CHKD","@AIDHC","@BannerChildrens","@BaptistOnline"), stringsAsFactors = FALSE)


for(handle in handles$Twitter.Handle) {

  result <- search_tweets(handle, n = 3500 , include_rts = FALSE,retryonratelimit = TRUE)

  if(length(result) != 0){
    result$`Twitter Handle` <- handle
    result$Source <- "Search"

    df_name <- paste0(tolower(substring(handle, 2)),"_search")

    if(exists(df_name)) {
      assign(df_name, unique(rbind(get(df_name), result)))
    } else {
      assign(df_name, result)
    }
  }
}