Is there a parameter I can pass to delayed_job that will prevent it from deleting completed jobs from the delayed_jobs table?
kind of like the destroy_failed_jobs but for completed jobs
any ideas?
It doesn't appear so. From the README: https://github.com/tobi/delayed_job
By default, it will delete failed jobs (and it always deletes successful jobs). If you want to keep failed jobs, set Delayed::Job.destroy_failed_jobs = false. The failed jobs will be marked with non-null failed_at.
You would probably need to hook into its destroy method such that it copies the job to another, separate table that keeps the list of completed jobs for you, or to simply log which jobs have been done to a file, if a log is all you need.
What you don't want is to leave the jobs in the delayed_jobs table, for a couple of reasons. First, because delayed_jobs uses that table as it's TODO list - you want it to only be things that still need to be done. Second, if you hacked it to keep all jobs in the same table, then the delayed_jobs table only grows, which would slow down the processing of delayed_jobs over time, as the query to find jobs that have not yet been completed would have to filter out those that have.