RoR: multiple calls in a row to the same long-time-response controller

Question

Update:

Read "Indicate to an ajax process that the delayed job has completed" before if you have the same problem. Thanks Gene.

---

I have a problem with concurrency. I have a controller scraping a few web sites, but each call to my controller needs about 4-5 seconds to respond.

So if I call 2 (or more) times in a row, the second call needs wait for the first call before starting.

So how I can fix this problem in my controller? Maybe with something like EventMachine?

Update & Example:

application_controller.rb

def func1
    i=0
    while i<=2
        puts "func1 at: #{Time.now}"
        sleep(2)
        i=i+1
    end
end

def func2
    j=0
    while j<=2
        puts "func2 at: #{Time.now}"
        sleep(1)
        j=j+1
    end
end

whatever_controller.rb

puts ">>>>>>>> Started At #{Time.now}"
  func1()
  func2()
puts "End at #{Time.now}"

So now I need request http://myawesome.app/whatever several times at the same times from the same user/browser/etc.

I tried Heroku (and local) with Unicorn but without success, this is my setup:

• unicorn.rb http://pastebin.com/QL0wdGx0
• Procfile http://pastebin.com/RrTtNWJZ
• Heroku setup https://www.dropbox.com/s/wxwr5v4p61524tv/Screenshot%202014-02-20%2010.33.16.png

Requirements:

• I need a RESTful solution. This is API so I need to responds JSON

More info: I have right now 2 cloud servers running.

• Heroku with Unicorn
• Engineyard Cloud with Nginx + Panssenger

Answer 1

For any long response time controller function, the delayed job gem is a fine way to go. While it is often used for bulk mailing, it works as well for any long-running task.

Your controller starts the delayed job and responds immediately with a page that has a placeholder - usually a graphic with a progress indicator - and Ajax or a timed reload that updates the page with the full information when it's available. Some information on how to approach this is in this SO article.

Not mentioned in the article is that you can use redis or some other memory cache to store the results rather than the main database.