The Sidekiq documentation recommends against having more than a few queues. I'm considering attempting to migrate a Resque setup that currently uses 107 queues.
Can anyone provide a more detailed explanation of why I shouldn't use Sidekiq with 107 queues? The above linked documentation just says that Sidekiq is "is not designed to work well with dozens of queues."
Rearchitecting the existing application isn't out of the question, but it makes Sidekiq a harder sell for us.
Sidekiq has a Fetcher which pulls jobs out of queues in Redis.
Your setup shouldn't be too bad for Sidekiq with the stock Fetcher but Sidekiq Pro's optional reliable Fetcher will pound Redis 107 times per second per process because Redis doesn't have a way to reliably block on multiple queues. That kind of heavy polling will kill your scalability.
So you can do that with Sidekiq. You can do that with Sidekiq Pro. Just don't turn on reliable fetch until you get your queue count down.