Sidekiq - Send job to Deadset directly without retrying

Question

How to send a job to the Deadset?

I've been searching around and I see some other developers that want to Notify to their exception handling systems when a job fails, for example if the job connects to an unreliable third party API.

In my case I would like to send the job to the deadset right away instead of just discarding it or waiting around 20 days.I want A human kind to have to click the Retry or Delete button based on the failure reason in the Deadset UI.

So if I have this Russian Roulette job that would retry if feeling lucky or go to the graveyard (aka Deadset) if not, how would be a proper way to do it?

It has come to my mind to set the retry: 0 in the sidekiq_options but that wouldn't be the case always. I think one way would be to use the retry method and pass somehow the sidekiq_options retry: 0 so next time it retries it self then it goes to the Deadset.

class RussianRoulette < ActiveJobBase

  sidekiq_options retry: 10

  discard_on SomeException do |job, exception|
    puts "You are dead but somebody will check your corpse!"
    # How to send to deadset?
  end

  def perform(args)
    if feeling_lucky? # If this method fails it retries up to 10 times
      puts "You got another chance!"
      retry_job
    else
      puts "You are dead!"
      # What to do?:
      # throw :abort # No, because we want to send it to the deadset so somebody can erase or retry it
      # raise SomeException # Could be, but then how to send to deadset?
      # self.class.perform_later(args, {sidekiq_options: {retry: 0}}) # Is this even possible?
    end
  end
end

The closest I've achieved is to push directly in the Deadset using the Sidekiq API, from the source I see that it expects a 'message'; I've tried to push the job as a JSON but in the WebUI it seems the message parameter is not correct.

Sidekiq::DeadSet.new.kill(job.to_json) # Also tried job.arguments.to_json

Update:

I have followed the logic of the Kill button in the WebUI and I see there is a kill method that is called to Job instance (or not?) but I see it's not available when I call it manually on console.

(byebug) job.kill
*** NoMethodError Exception: undefined method `kill' for #<RussianRoulette:0x00007faf520805a8>

Answer 1

I ended up injecting directly in the Deadset the record by just using the params of my original job. I captured the exception (using a custom exception) and the rescue_from method.

class MyJob < ActiveJob::Base

  rescue_from(CustomException) do |exception|
    params = self.arguments
    params[0]['_aj_symbol_keys'] = params[0].keys.map(&:to_s)
    job = {
      "error_message": exception.message, 
      "error_class": exception.class.name, 
      "jid": SecureRandom.hex(12),
      "wrapped": self.class.name, 
      "created_at": Time.now.to_f, 
      "enqueued_at": Time.now.to_f, 
      "failed_at": Time.now.to_f, 
      "retry_count": 0,
      "args": [
        {
          "job_class": self.class.name,
          "job_id": SecureRandom.hex(12),
          "provider_job_id": nil,
          "priority": nil,
          "arguments":  params,
          "executions": 0,
          "queue_name": "default",
          "exception_executions": {},
          "locale": "en",
          "timezone": "UTC",
          "enqueued_at": Time.now.to_f
        }
      ],
      "retry": 0,
      "queue": "default",
      "class": "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper"
    }
    Sidekiq::DeadSet.new.kill(job.to_json) 
  end

  def perform(params)
    raise CustomException.new("Here is a custom exception") if whatever
  end
end

So when i need to send a Job to the morgue directly inside my job logic i just raise an exception and thats it, the rescue_from block will capture it and create it in the morgue.

After this it shows up in my Morgue:

And some human can analyze it and delete or retry it. In my case this was a web scraper, so if the HTML of my target site goes invalid then a developer has to update the logic and we have to retry with the original arguments once the developer pushes the updated scraping code.