Laravel custom Artisan command stops at 50 percent completion

Question

So I have a custom artisan command that I wrote to slug a column and store it into a new column. I have a progress bar implemented, and for some reason, when the command reaches 50% completion it jumps to 100%. The issue is that it has only executed the code on half of the data.

I am using the chunk() function to break the data into chunks of 1,000 rows to eliminate memory exhaustion issues. This is necessary because my dataset is extremely large.

I have looked into my PHP error logs, MySQL error logs, and Laravel logs. I can't find any error or log line pertaining to this command. Any ideas on a new place to even start looking for the issue.

$jobTitles = ModelName::where($columnName, '<>', '')
            ->whereNull($slugColumnName)
            ->orderBy($columnName)
            ->chunk(1000, function($jobTitles) use($jobCount, $bar, $columnName, $slugColumnName)
        {
            foreach($jobTitles as $jobTitle) 
            {
                $jobTitle->$slugColumnName = Str::slug($jobTitle->$columnName);
                $jobTitle->save();
            }
            $bar->advance(1000);
        });
        $bar->finish();

Answer 1

What's happening is the whereNull($slugColumnName) in combination with the callback setting the $slugColumnName is leading to missed results on subsequent loops.

The order of events is something like this:

• Get first set of rows: select * from table where column is null limit 100;
• For each of the rows, set column to a value.
• Get next set of rows: select * from table where column is null limit 100 offset 100;.
• Continue and increase the offset until no more results.

The problem here is that after the second step you have removed 100 results from the total. Say you begin with 1000 total rows, by the second query you now only have 900 matching rows.

This causes the offset to be seemingly skipping an entire chunk by starting at row 100, when the first 100 rows have not been touched yet.

For more official documentation, please see this section of this section on chunking.

I have not tested this to verify it works as expected for your use-case, but it appears that using chunkById will account for this issue and correct your results.