Ruby and Rails Async

Question

I need to perform long-running operation in ruby/rails asynchronously. Googling around one of the options I find is Sidekiq.

class WeeklyReportWorker
  include Sidekiq::Worker

  def perform(user, product, year = Time.now.year, week = Date.today.cweek)
    report = WeeklyReport.build(user, product, year, week)
    report.save
  end
end

# call WeeklyReportWorker.perform_async('user', 'product')

Everything works great! But there is a problem.

If I keep calling this async method every few seconds, but the actual time heavy operation performs is one minute things won't work.

Let me put it in example.

5.times { WeeklyReportWorker.perform_async('user', 'product') }

Now my heavy operation will be performed 5 times. Optimally it should have performed only once or twice depending on whether execution of first operaton started before 5th async call was made.

Do you have tips how to solve it?

Answer 1

Here's a naive approach. I'm a resque user, maybe sidekiq has something better to offer.

def perform(user, product, year = Time.now.year, week = Date.today.cweek)
  # first, make a name for lock key. For example, include all arguments
  # there, so that another perform with the same arguments won't do any work
  # while the first one is still running
  lock_key_name = make_lock_key_name(user, product, year, week)
  Sidekiq.redis do |redis| # sidekiq uses redis, let us leverage that
    begin
      res = redis.incr lock_key_name
      return if res != 1 # protection from race condition. Since incr is atomic, 
                         # the very first one will set value to 1. All subsequent
                         # incrs will return greater values.
                         # if incr returned not 1, then another copy of this 
                         # operation is already running, so we quit.

      # finally, perform your business logic here
      report = WeeklyReport.build(user, product, year, week)
      report.save
    ensure
      redis.del lock_key_name # drop lock key, so that operation may run again.
    end
  end
end