Search code examples
phpmutexfile-lockingflocktest-and-set

PHP flock() for read-modify-write does not work

I have a log file maintained by a PHP script. The PHP script is subject to parallel processing. I cannot get the flock() mechanism to work on the log file: in my case, flock() does not prevent the log file shared by PHP scripts running in parallel from being accessed at the same time and being sometimes overwritten.

I want to be able to read a file, do some processing, modify the data and write back without the same code running in parallel on the server doing the same at the same time. The read modify write has to be in sequence.

On one of my shared hostings (OVH France), it does not work as expected. In that case, we see that the counter $c has the same value in different iframes, which should not be possible if the lock works as expected, which it does on an other shared hosting.

Any suggestions to make this work, or for an alternative method?

Googling "read modify write" php or fetch and add or test and set did not provide useful information: all solutions are based on a working flock().

Here is some standalone running demo code to illustrate. It generates a number of parallel requests from the browser to the server and displays the results. It is easy to visually observe a disfunction: if your webserver does not support flock() like one of mine, the counter value and the number of log lines will be the same in some frames.

<!DOCTYPE html>
<html lang="en">
<title>File lock test</title>
<style>
iframe {
    width: 10em;
    height: 300px;
}
</style>
<?php
$timeStart = microtime(true);
if ($_GET) { // iframe
    // GET
    $time = $_GET['time'] ?? 'no time';
    $instance = $_GET['instance'] ?? 'no instance';

    // open file
    // $mode = 'w+'; // no read
    // $mode = 'r+'; // does not create file, we have to lock file creation also
    $mode = 'c+'; // read, write, create
    $fhandle = fopen(__FILE__ .'.rwtestfile.txt', $mode) or exit('fopen');
    // lock
    flock($fhandle, LOCK_EX) or exit('flock');
    // start of file (optional, only some modes like require it)
    rewind($fhandle);
    // read file (or default initial value if new file)
    $fcontent = fread($fhandle, 10000) or ' 0';
    // counter value from previous write is last integer value of file
    $c = strrchr($fcontent, ' ') + 1;
    // new line for file
    $fcontent .= "<br />\n$time $instance $c";
    // reset once in a while
    if ($c > 20) {
        $fcontent = ' 0'; // avoid long content
    }
    // simulate other activity
    usleep(rand(1000, 2000));
    // start of file
    rewind($fhandle);
    // write
    fwrite($fhandle, $fcontent) or exit('fwrite');
    // truncate (in unexpected case file is shorter now)
    ftruncate($fhandle, ftell($fhandle)) or exit('ftruncate');
    // close
    fclose($fhandle) or exit('fclose');
    // echo
    echo "instance:$instance c:$c<br />";
    echo $timeStart ."<br />";
    echo microtime(true) - $timeStart ."<br />";
    echo $fcontent ."<br />";
} else {
    echo 'File lock test<br />';
    // iframes that will be requested in parallel, to check flock
    for ($i = 0; $i < 14; $i++) {
        echo '<iframe src="?instance='. $i .'&time='. date('H:i:s') .'"></iframe>'."\n";
    }
}

There is a warning about flock() limitations in the PHP: flock - Manual, but it is about ISAPI (Windows) and FAT (Windows). My server configuration is:
PHP Version 7.2.5
System: Linux cluster026.gra.hosting.ovh.net
Server API: CGI/FastCGI

Solution

  • A way to do an atomic test and set instruction in PHP is to use mkdir(). It is a bit strange to use a directory for that instead of a file, but mkdir() will create a directory or return a false (and a suppressile warning) if it already exists. File commands like fopen(), fwrite(), file_put_contents() do not test and set in one instruction.

    <?php
// lock
$fnLock = __FILE__ .'.lock'; // lock directory filename
$lockLooping = 0; // counter can be used for tuning depending on lock duration
do {
    if (@mkdir($fnLock, 0777)) { // mkdir is a test and set command
        $lockLooping = 0;
    } else {
        $lockLooping += 1;
        $lockAge = time() - filemtime($fnLock);
        if ($lockAge > 10) {
            rmdir($fnLock); // robustness, in case a lock was not erased                
        } else {
            // wait without consuming CPU before try again
            usleep(rand(2500, 25000)); // random to avoid parallel process conflict again
        }
    }
} while ($lockLooping > 0);

// do stuff under atomic protection
// don't take too long, because parallel processes are waiting for the unlock (rmdir)

$content = file_get_contents($protected_file_name);  // example read
$content = $modified_content; // example modify
file_put_contents($protected_file_name, $modified_content); // example write

// unlock
rmdir($fnLock);