Search code examples
ruby-on-railsemailqueuesidekiq

Prioritizing jobs on Sidekiq


Imagine a simple dating app, which provides you a potential match every day. When users first sign up, the following happens -

  1. They are sent an email - "Welcoming to the Dating app!"

  2. A custom matching job kicks off to find them a match for the day and sends them an email - "You have a match!"

If I choose to mass-enroll several thousand people, the job queue will be populated with several thousand jobs of both #1 and #2. It's important that they receive the email first before getting a match, since otherwise it makes for a confusing experience.

I have -

# config/sidekiq.rb
:queues:
  - high_priority
  - default
  - low_priority

As per this page, those queues should process in the listed order.

  1. How does this prioritization work? Is the entire high priority queue drained before picking an item of the default or low_priority queues? Or is it just checked more often, still allowing some lower priority jobs to be processed at the same time?

  2. As a follow-up to #1, is there a way to ensure that for a specific user, they receive the welcome email before they receive their match email? The former will be in the high_priority queue and the latter in the default priority, but there's no guarantee either queue is in any particular order for users.

Thanks!


Solution

  • Each time Sidekiq looks for a job in Redis, it will check high_priority first, then default, then low_priority. The jobs in default will just sit there as long as there are jobs in high_priority.