Search code examples
phpgearman

Adding a new job with GearmanManager?

I'm still new to the whole Gearman and GearmanManager cycle. I have a working server and have verified my jobs run if they're already in my queue (MySQL table) when the server starts. However, I need to be able to add a new job to the queue from PHP, and from inside of the worker if possible.

Right now I have a job that I will be creating on the deployment of our new codebase. This will be the first job to run and it's purpose is to gather some data for reports and store it.

This needs to be run every hour on the hour, so I want to utilize the when_to_run column. I've been mulling over the documentation for Gearman but I'm still confused on how I'm actually supposed to add a job to the queue.

I've tried running:

<?php
$gm = new GearmanClient;
$gm->addServer();
$gm->doBackground('Metadata_Ingest_Report', '', com_create_guid());

On a side note, yes, I do have php-pecl-uuid installed.

The above code just hangs and doesn't do anything. No job is added to the DB, nothing happens.

This is due to me not fully understanding how a job gets sent, and I'm doing my best to RTM, but I'm not having any luck.

So if there is something you can point me to, or if someone has some time to explain how I'm supposed to setup and add jobs to the MySQL queue so GearmanManager's workers pick them up that would be awesome.

Edit: So it appears that you have to call $gm->addServer('127.0.0.1'). According to the documentation 127.0.0.1 is supposed to be the default, but that does not appear to be the case running PHP 5.4.11. I can now get the tasks to run if I call $gm->runTasks() after $gm->addTask(). I would expect to just have to call $gm->addTask() and the task would be added to the DB and GearmanManager would see it and spool it up. Still digging...

Best regards,
Andrew

Solution

  • So it appears that the when_to_run functionality is not exposed on pecl-gearman. Because this, we are unable to schedule jobs for the future using their built in methods. It also appears that the library does not create the DB records like it should (I'd assume this may actually be Gearmand not offloading the jobs to the DB before they're run.

    To get around this we have decided to do the following.

    Scheduling Future Jobs

    1. Manually INSERT the job into gearman_queue.
    2. Run a CRON every minute to ping the queue table and load jobs that have a when_to_run <= time()
    3. Fire those jobs via addTask($function, $payload) and runTasks(). The $payload contains the UUID from the DB as well.
    4. GearmanManager picks up the job and hands off the payload to their respective workers.
    5. Worker runs, then on completion, removes the item from the DB with a DELETE.

    Running Job Immediately

    1. Manually INSERT the job into gearmand_queue with a when_to_run of NULL.
    2. Run addTask($function, $payload) and runTasks(). The $payload contains the UUID from the DB as well.
    3. GearmanManager picks up the job and hands off the payload to their respetive workers.
    4. Worker runs, then on completion, removes the item from the DB with a DELETE.

    Conclusion

    Gearmand Job Server, GearmanManager, and pecl-gearman all seem to be out of sync when it comes to what is supported and how it's done. For the most part I think this issue lays within the core of pecl-gearman talking to Gearmand.

    I have also opened a feature request on the pecl-gearman project for when_to_run: https://bugs.php.net/bug.php?id=64120