Search code examples
phpqueuebeanstalkd

Beanstalkd custom task delay for 2nd and 3rd retry


I use Beanstalkd (with - https://github.com/udokmeci/yii2-beanstalk) for queues. But I need to set custom delay for next retry,

1st - with no delay 2nd - in a hour 3th - in 24 hrs

Is it possible to implement this with Beanstalkd?

public function actionCron($job)
{
    $sentData = $job->getData();
    try {

        // I need to setup custom delay there
        // 1 hour after 1st retry
        // 24 hrs after 2nd retry

        return self::DELAY;

    } catch (\Exception $e) {
        //If there is anything to do.
        fwrite(STDERR, Console::ansiFormat($e . "\n", [Console::FG_RED]));

        // you can also bury jobs to examine later
        return self::BURY;
    }
}

Solution

  • It's quite possible - by asking the Beanstalkd server for stats about the job.

    From the Beanstalk protocol.txt file:

    stats-job <id>\r\n
    

    The data returned includes the number of times the job has previously been reserved, released, and from which tube-name etc. You can use whatever data is most appropriate to re-queue it as required.

    Since you can't change the data to delay it, only create a new job, an option is to put it into a new queue for a 2nd or 3rd attempt, with suitable initial delay, and then if the job has come from a retry-queue (after the 1st, or 2nd attempt), alter the initial delay before it is run again.

    Since you can attempt to get a job from multiple queues, you'll just have your code watch the main queue, and also the 2nd/3rd-attempt tubes and see what they are given, processing as appropriate.