Will Sidekiq perform_in work even with a constantly full queue?

Question

For example, if I want a job to be performed in an hour and do 'MyWorker.perform_in 1.hour', but there are so many in the queue that even in an hour the queue is still full, will the job that I want to perform in an hour perform at the desired time or will it get enqueued at the end of the queue?

Answer 1

From the Sidekiq repository:

The Poller checks Redis every N seconds for jobs in the retry or scheduled set have passed their timestamp and should be enqueued. If so, it just pops the job back onto its original queue so the workers can pick it up like any other job.

And just a bit farther down:

def start
  @thread ||= safe_thread("scheduler") do
    initial_wait

    while !@done
      enqueue
      wait
    end
    Sidekiq.logger.info("Scheduler exiting...")
  end
end

It appears that the scheduler will enqueue the job at the scheduled time, not force the worker to begin executing the job at that time.

One possible solution is to have a custom queue for the scheduled jobs and one for unscheduled jobs. That way if the scheduled job queue is full, the other queue can still process jobs. More information on queues is available in the Sidekiq wiki, where this type of setup is described in greater detail.