Search code examples
sidekiq

Give Sidekiq queue a maximum size limit

Recently one queue for reporting exceptions got overflowed by jobs, causing Redis to use too much memory and finally crash.

I want to give some Sidekiq queue a maximum job option. Jobs should be dropped if the queue is full. Is there an option or gem to provide this functionality?

Solution

  • There's nothing out there today that I know of that provides real queue backpressure. Some ideas, hope this helps:

    Use Inspeqtor to monitor your Redis instance:

    https://github.com/mperham/inspeqtor/wiki/Daemon-Specific-Metrics#redis

    Monitor your queue backlog:

    https://github.com/mperham/sidekiq/wiki/Monitoring#monitoring-queue-backlog