Search code examples
phplaravelcronjobs

Laravel Job Duplication

I'm currently experiencing an issue with job duplication in my Laravel application. I have a job that I dispatch once dispatch(new SendFirebaseNotificationJob($data));, but while monitoring the artisan queue:work --sleep=3 --tries=0 --backoff=3 --max-time=3600 command, I sometimes see that the job handle has been triggered multiple times (4 or 6 times).


  2023-05-21 23:31:24 App\Jobs\SendFirebaseNotificationJob ..... 108.77ms DONE
  2023-05-21 23:31:57 App\Jobs\SendFirebaseNotificationJob ........... RUNNING

  2023-05-21 23:31:57 App\Jobs\SendFirebaseNotificationJob ..... 152.49ms DONE
  2023-05-21 23:32:24 App\Jobs\SendFirebaseNotificationJob ...........  2023-05-21 23:32:24 App\Jobs\SendFirebaseNotificationJob ........... RUNNING

 RUNNING

  2023-05-21 23:32:24 App\Jobs\SendFirebaseNotificationJob ..... 209.73ms DONE
  2023-05-21 23:32:24 App\Jobs\SendFirebaseNotificationJob ..... 444.06ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
 ........... RUNNING
Processing job with UID: 764 <- //The same job start mutabile times!
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
Processing job with UID: 764
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 131.32ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 131.92ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob .....  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 159.31ms DONE
 155.82ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 156.36ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 156.24ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 156.36ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 156.58ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 192.53ms DONE
  2023-05-21 23:37:24 App\Jobs\SendFirebaseNotificationJob ..... 237.49ms DONE
  2023-05-21 23:41:28 App\Jobs\SendFirebaseNotificationJob ........... RUNNING
  2023-05-21 23:41:28 App\Jobs\SendFirebaseNotificationJob ........... RUNNING

I've tried to identify the cause of the issue, but I haven't been able to find a solution so far.

I would appreciate any advice or guidance on how to resolve this issue. If anyone has experienced a similar problem or has any suggestions on what I could try, I would be grateful for any input.

Thanks in advance for your help.

Checking if the job is idempotent: I reviewed the job code to ensure that it can not be executed multiple times, i deleted the job request from the DB so it will not be dispatched.

However, I am still experiencing job duplication in my Laravel application. I expected the job to be executed only once

Solution

  • You can prevent dispatching job multiple times and job overlaps with WithoutOverlapping job middleware - https://laravel.com/docs/10.x/queues#preventing-job-overlaps

    It is good practice to use it on jobs that you need to be sure are dispatched only once.