PHP bulk processing solution

Question

I'm looking for a PHP component for asynchronous data processing.

Basically what I need is to display a page with a progress bar that's refreshed with javascript which displays the progress on some data processing.

On the backend you'll define your data process limit. This is the start, end and function to call for processing individual items.

There are plenty of solutions for this on CMS and frameworks. I'm looking for something in raw PHP that I can include in my application.

Answer 1

I did something similar not too long ago. I wrote a function that logs the progress to a text file as a JSON object. Then I wrote a PHP function that returns that JSON object to the browser at certain intervals as requested by jQuery.

My PHP code looks similar to this:

function logProgress($task, $status, $progress) {


$basedir = "/var/www/" . SITE_ROOT . "/";
$log_file = $basedir . "logs/progress.log";

$logFileContent = file_get_contents($mrp_log_file);

if($logFileContent){
    $logFileArray = json_decode($logFileContent, TRUE);
} else {
    $logFileArray = array();
}

$logFileArray[$task]=array('task'=>$task,'status'=>$status,'progress'=>$progress);

$logFile = fopen($log_file, 'w+') or error_log("Failed to open progress file $mrp_log_file for writing");

fwrite($logFile, json_encode($logFileArray));

fclose($logFile);

}

Retrieving the data is as simple as this:

function readProgressLog() {
//Returns a JSON object stored in the progress log.
$basedir = "/var/www/" . SITE_ROOT . "/";
$log_file = $basedir . "logs/progress.log";

$logFileContents = file_get_contents($log_file);

return $logFileContents;

}

From jQuery, you would make two AJAX calls, one to initiate your process, and one to poll the text file. My javascript for the polling call looks like this:

function updateProgress() {
var data = {
    action:'getProgressUpdate'};
var settings = {success: function(json){
    var done = false;

    if(json!=null) {
        //Put your code to update the progress bar here.
                    //I look for a JSON property called Done to flag the process as completed.                      
        if(json.Done==null) {
            var t2 = setTimeout("updateProgress()", 1000);
        } else {
            clearTimeout(t2);
            done = true;
            clearProgressLog();
        }
    } else {
         var t2 = setTimeout("updateProgress()", 1000);
    }
},
data:data,
cache:false,
type: 'POST',
dataType:"json"};

$.ajax('/ajax/polling.ajax.php', settings);

}

One thing I noticed is that you should make sure your polling AJAX call uses a different PHP file than your process AJAX call, otherwise your polling call won't finish until the process call is finished.