I am using Rails / Delayed Jobs, with Rails 4.1 on Heroku. I've noticed that my jobs take anywhere from 1 second to 10 seconds to actually start. Once they start, they run pretty fast.
How can I speed that up?
I am calling them with my_thing.delay.run!
I do have some other ongoing jobs, but there's not that many of them, so it doesn't seem like that would be the cause. It just seems like a lagginess in how often it checks to run jobs.
I think you want to configure Delayed::Worker.sleep_delay which is hinted at in the delayed job README. If delayed job can't find a job then it sleeps for this many seconds before looking again. The default sleep is for 5 seconds.
So, you might set the following in config/initializers/delayed_job.rb to only sleep for 2 seconds between queries for pending jobs.
Delayed::Worker.sleep_delay = 2
Obviously the trade-off is the more frequent polling for jobs when nothing is going on.
Also, if you're not wedded to delayed job then you might find resque or particularly sidekiq would probably process your jobs quicker than delayed job.