Search code examples
sidekiq

Limit number of threads per sidekiq queue

Is there a way for me to limit the number of maximum threads per sidekiq queue?

For example:

We have a daily job that takes quite a lot of time to get processed. On the one hand we can say let's put 20 threads and get it done faster. Well, no - that clogs up our database.

What I want is to have something similar to: DailyJobsQueue - 10 threads max, regularJobsQueue - no limit of threads (the rest of available threads - not to say they will be used not in peak times).

I've seen I can give priority to threads that changes the polling for jobs.. But that's not what I need.

Any ideas?

Solution

  • Sidekiq does not allow you to limit concurrency.

    I would put the daily jobs in a low priority queue. Normal work will still get processed quickly and the daily jobs will be processed by any leftover threads.