Limit the amount of workers per queue in Sidekiq

Question

I've been trying to limit the amount of workers per queue using the sidekiq-limit_fetch gem, and Sidekiq seems to "see" the imposed limits in the log but when I watch the workers the limits are ignored.

Here's the part from the log where Sidekiq sees the limits:

2013-04-02T05:47:19Z 748 TID-11ilcw DEBUG: {:queues=>
    ["recommendvariations",
     "recommendvariations",
     "recommendvariations",
     "recommendphenotypes",
     "recommendphenotypes",
     "recommendphenotypes",
     "preparse",
     "preparse",
     "preparse",
     "parse",
     "parse",
     "parse",
     "zipgenotyping",
     "zipgenotyping",
     "zipfulldata",
     "deletegenotype",
     "fitbit",
     "frequency",
     "genomegov",
     "mailnewgenotype",
     "mendeley_details",
     "mendeley",
     "pgp",
     "plos_details",
     "plos",
     "snpedia",
     "fixphenotypes"],
   :concurrency=>5,
   :require=>".",
   :environment=>"production",
   :timeout=>8,
   :profile=>false,
   :verbose=>true,
   :pidfile=>"/tmp/sidekiq.pid",
   :logfile=>"./log/sidekiq.log",
   :limits=>
    {"recommendvariations"=>1,
     "recommendphenotypes"=>1,
     "preparse"=>2,
     "parse"=>2,
     "zipgenotyping"=>1,
     "zipfulldata"=>1,
     "fitbit"=>3,
     "frequency"=>10,
     "genomegov"=>1,
     "mailnewgenotype"=>1,
     "mendeley_details"=>1,
     "mendeley"=>1,
     "pgp"=>1,
     "plos_details"=>1,
     "plos"=>1,
     "snpedia"=>1,
     "fixphenotypes"=>1},
   :strict=>false,
   :config_file=>"config/sidekiq.yml",
   :tag=>"snpr"}

and here's the sidekiq.yml. Judging from the web-interface of sidekiq the limits are ignored - right now, I got 2 workers on the "recommendvariations"-queue but that should be 1.

I start the workers over bundle exec sidekiq -e production -C config/sidekiq.yml.

Has anyone else ever encountered this?

Answer 1

Did you try to set the limit in a sidekiq.rb initializer file?

Like this:

Sidekiq::Queue['recommend'].limit = 1

It worked for me.