Make POST request with large file in body for PHP 5.5+

Question

There is an API I need to work with from my PHP application. One endpoint receives a file to upload as a body of the POST request. The uploaded file can be rather huge (up to 25GB). The endpoint returns a simple JSON content with either 200 OK or different other status codes.

Example request may look like this:

POST /api/upload HTTP/1.1
Host: <hostname>
Content-Type: application/octet-stream
Content-Length: 26843545600
Connection: close

<raw file data up to 25 GB>

Basically, I need to write a method that will perform such request without killing the server.

I tried to find any reasonable implementation but, from what I can see, both cURL and non-cURL (stream_context_create) methods require string request body, which may exhaust server memory.

Is there any simple way to achieve this without writing a separate socket transport layer?

Answer 1

Since no better options were found, I went for a default solution with fsockopen.

Here is the full source code of the utility function that will perform an HTTP request with low memory consumption. As the data parameter it can accept string, array and SplFileInfo object.

/**
 * Performs memory-safe HTTP request.
 *
 * @param string $url       Request URL, e.g. "https://example.com:23986/api/upload".
 * @param string $method    Request method, e.g. "GET", "POST", "PATCH", etc.
 * @param mixed $data       [optional] Data to pass with the request.
 * @param array $headers    [optional] Additional headers.
 *
 * @return string           Response body.
 *
 * @throws Exception
 */
function request($url, $method, $data = null, array &$headers = []) {
    static $schemes = [
        'https' => ['ssl://', 443],
        'http'  => ['', 80],
    ];

    $u = parse_url($url);
    if (!isset($u['host']) || !isset($u['scheme']) || !isset($schemes[$u['scheme']])) {
        throw new Exception('URL parameter must be a valid URL.');
    }

    $scheme = $schemes[$u['scheme']];
    if (isset($u['port'])) {
        $scheme[1] = $u['port'];
    }

    $fp = @fsockopen($scheme[0] . $u['host'], $scheme[1], $errno, $errstr);
    if ($fp === false) {
        throw new Exception($errstr, $errno);
    }

    $uri = isset($u['path']) ? $u['path'] : '/';
    if (isset($u['query'])) {
        $uri .= '?' . $u['query'];
    }

    if (is_array($data)) {
        $data = http_build_query($data);
        $headers['Content-Type'] = 'application/x-www-form-urlencoded';
        $headers['Content-Length'] = strlen($data);
    } elseif ($data instanceof SplFileInfo) {
        $headers['Content-Length'] = $data->getSize();
    }

    $headers['Host'] = $this->host;
    $headers['Connection'] = 'close';

    fwrite($fp, sprintf("%s /api%s HTTP/1.1\r\n", $method, $uri));
    foreach ($headers as $header => $value) {
        fwrite($fp, $header . ': ' . $value . "\r\n");
    }
    fwrite($fp, "\r\n");

    if ($data instanceof SplFileInfo) {
        $fh = fopen($data->getPathname(), 'rb');
        while ($chunk = fread($fh, 4096)) {
            fwrite($fp, $chunk);
        }
        fclose($fh);
    } else {
        fwrite($fp, $data);
    }

    $response = '';
    while (!feof($fp)) {
        $response .= fread($fp, 1024);
    }
    fclose($fp);

    if (false === $pos = strpos($response, "\r\n\r\n")) {
        throw new Exception('Bad server response body.');
    }

    $headers = explode("\r\n", substr($response, 0, $pos));
    if (!isset($headers[0]) || strpos($headers[0], 'HTTP/1.1 ')) {
        throw new Exception('Bad server response headers.');
    }

    return substr($response, $pos + 4);
}

Example usage:

$file = new SplFileObject('/path/to/file', 'rb');
$contents = request('https://example.com/api/upload', 'POST', $file, $headers);
if ($headers[0] == 'HTTP/1.1 200 OK') {
    print $contents;
}