Search code examples
laravellaravel-5jobs

How can I troubleshoot silently failing queued jobs?


I have a job that is dispatched with two arguments - path and filename to a file. The job parses the file using simplexml, then makes a notice of it in the database and moves the file to the appropriate folder for safekeeping. If anything goes wrong, it moves the file to another folder for failed files, as well as creates an event to give me a notification.

My problem is that sometimes the job will fail silently. The job is removed from the queue, but the file has not been parsed and it remains in the same directory. The failed_jobs table is empty (I'm using the database queue driver for development) and the failed() method has not been triggered. The Queue::failing() method I put in the app service provider has not been triggered either - I know, since both of those contain only a single log call to check whether they were hit. The Laravel log is empty (it's readable and Laravel does write to it for other errors - I double-checked) and so are relevant system log files such as e.g. php's.

At first I thought it was a timeout issue, but the queue listener has not failed or stopped, nor been restarted. I increased the timeout to 300 seconds anyway, and verified that all of the "[datetime] Processed: [job]" lines the listener generates were well within that timespan. Php execution times etc. are also far longer than required for this job.

So how on earth can I troubleshoot this when the logs are empty, nothing appears to fail, and I get no notification of what's wrong? If I queue up 200 files then maybe 180 will be processed and the remaining 20 fail silently. If I refresh the database + migrations and queue up the same 200 again, then maybe 182 will be processed and 18 will fail silently - but they won't necessarily be the same.

My handle method, simplified to show relevant bits, looks as follows:

public function handle()
{
    try {
        $xml = simplexml_load_file($this->path.$this->filename);
        $this->parse($xml);
        $parsedFilename = config('feeds.parsed path').$this->filename;
        File::move($this->path.$this->filename, $parsedFilename);
    } catch (Exception $e) {
        // if i put deliberate errors in the files, this works fine
        $errorFilename = config('feeds.error path').$this->filename;
        File::move($this->path.$this->filename, $errorFilename);
        event(new ParserThrewAnError($this->filename));
    }
}

Solution

  • Okay, so I still have absolutely no idea why, but... after restarting the VM I have tested eight times with various different files and options and had zero problems. If anyone can guess the reason, feel free to reply and I'll accept your answer if it sounds reasonable. For now, I'll mark my own answer as correct once I can, in case somebody else stumbles across this later.