Search code examples
phpphp-ziparchive

PHP ZipArchive extractTo() stalls with large files


I am working on a script that uses the extractTo method of ZipArchive to unzip some pretty large files (some are over 10G). Everything works fine for small files, but I have been testing it with files that are ~4G and noticed that unzip works up to a point, then it stops actually unzipping. However, the PHP script appears to be running. No errors or exceptions are thrown. From a terminal I sit in the folder and keep typing ls -la to watch the size of the extracted file grow. It does so for a while, then stops and the script continues to load (watching via browser and via top). The script will then run for the specified timeout period (I set to 3600) and throw a time-out error. The box is running Centos 6.6, 16G of RAM, plenty of processing power, and plenty of disc space. Seems to be crashing at 5011800064 bytes unzipped. Here are some select bits of my code:

set_time_limit(1200);
ini_set('memory_limit', '1024M');

$zip = new ZipArchive;

    $res = $zip->open($zipPath);

    if ($res === TRUE)
    {
        $extres = $zip->extractTo(
            $unzipPath, 
            $filesToExtract
        );

        $zip->close();
    }

Any help would be greatly appreciated. I am also curious to know if the extractTo() function tries to load the whole zip into memory? I have scoured the PHP documentation and cannot find anything relevant. All of the related posts either do not have answers and were not specific in their explanation of the problem.

Edit: Just to confirm, I have over 20G free and setting different memory limits for the script doesn't change the number of bytes unzipped.

Update: I have scoured httpd.conf, php.ini, and cannot find any settings that are prohibiting the unzip operations from working.


Solution

  • A traditional .zip archive is limited in size to 4GB:

    The maximum size for both the archive file and the individual files inside it is 4,294,967,295 bytes (232−1 bytes, or 4 GiB minus 1 byte) for standard .ZIP, and 18,446,744,073,709,551,615 bytes (264−1 bytes, or 16 EiB minus 1 byte) for ZIP64.

    To decompress larger archives using ZIP64 in PHP, you'll need to use PHP 5.6 that uses libzip 0.11.2.