Why am I getting "Too many terms specified in query" for calls within the Twitter API 100-user limitation?

Question

I'm using list/members/create_all and list/members/destroy_all which both claim a limitation of 100 users per call. I limit my calls to 90 users, but I still intermittently receive a Too many terms specified in query error ... I'll even intermittently get this error when I limit my calls to 40 users or less.

It seems to apply only to certain users that I make the call on behalf of: I can iterate over some users just fine with my 90-user limit, but some of the users I iterate over will error out with Too many terms specified in query unless I severely limit my calls (for example, 10 users at a time) ... but that severe limitation just presents a different problem - I hit Rate Limit when I try to make so many small calls for the user.

I'm iterating over each user in my database with User.all.each do |u|, and I'm creating a connection to Twitter for each user within that block with:

  client = Twitter::Client.new(
    :consumer_key => TWITTER_CONSUMER_KEY,
    :consumer_secret => TWITTER_CONSUMER_SECRET,
    :oauth_token => u.twitter_token,
    :oauth_token_secret => u.twitter_secret
  )

This is the code I use to remove users from a list:

  removing_from_list.each_slice(90) do |remove_ids|
    client.list_remove_members(list_id, remove_ids)
    sleep 2
  end

and this code to add users to a list:

  adding_to_list.each_slice(90) do |adding_ids|
    client.list_add_members(list_id, adding_ids)
    sleep 2
  end

I've experienced this issue both in use of the Twitter gem as well as by accessing the Twitter API directly with Rested (a Mac REST client).

Answer 1

I had also posted this question in the REST API v1.1 Developer Discussions, where I received a suggestion for the list_add_members portion of my issue. That suggestion was to make sure that the list wasn't going over 500 members, and that the user didn't have 20+ lists. I'm not sure how relevant it was, given what I found out, but it's certainly good advice nonetheless.

The way that I resolved this issue was by deleting & recreating the list itself, if I hit the error:

  removing_from_list.each_slice(99) do |remove_ids|
    begin
      client.list_remove_members(list_id, remove_ids)
      sleep 2
    rescue => error
      if error == "Too many terms specified in query"
        client.list_destroy(list_id)
        client.list_create('ListName', :description => 'List Description')
        list_id = client.list('ListName').id
      end
    end
  end

After this was done, the list behaved as expected. Also, I haven't had to delete/recreate a list more than once. So, I'm fairly confident in declaring that it was either an issue with the list itself or a user that happened to reside in the list before it was deleted/recreated.