Search code examples
ruby-on-railsrubysidekiq

Is there any 'group async wait' for sidekiq's tasks?

Like javascript's Promise.all can wait all requests finished and then do the following job.

For example, I have A, B, C tasks:

C needs A and B both finished to run and use the results calculated by A and B.

(Because 3 processes are all very expensive computation so they're put in sidekiq queue to run)

Solution

  • Use Sidekiq Batches which is available with Sidekiq Pro.

    Batches will let you orchestrate your jobs in this manner and let A and B run in parallel. After completion of batch, you can allocate a callback and trigger C. Read more about callbacks here.

    class MyCallback
  def on_success(status, options)
    # Initiate call for C
  end
end

batch = Sidekiq::Batch.new
batch.on(:success, MyCallback)
batch.jobs do
  [a, b].each { |job| GenericJobWorker.perform_async(job) }
end
puts "Just started Batch #{batch.bid}"