Search code examples
phpsymfonydoctrine-ormdoctrine-odm

Downloading CSV with StreamedResponse stops after several rows


I have a MongoDB collection with a few thousand entries which I would like to download as a CSV file.

I have some code which basically does the following; A controller method queries the database using Doctrine ODM for all records. The returned cursor is then fed to a StreamedResponse. In the StreamedResponse I loop over the cursor and output each record as a line in the CSV.

The following code works, and a file gets downloaded. Only, it contains not more than 60 rows. What could be the reason that it stops streaming at that point, while the cursor count indicates that there are 2670 results?

// Get all available bookings
$cursor = $this
        ->get('doctrine_mongodb')
        ->getRepository('FooBundle:Booking')
        ->getQueryBuilder()
        ->getQuery()
        ->execute();

$response = new StreamedResponse(function () use ($cursor) {

    $handle = fopen('php://output', 'r+');

    $headerPrinted = false;

    // Calling $cursor->count() here returns 2670

    foreach ($cursor as $result) {
        // Transform the Booking object to a flat array.
        $data = $this->constructRow($result);

        // Print CSV header
        if (!$headerPrinted) {
            fputcsv($handle, array_keys($data), ';', '"');
            $headerPrinted = true;
        }

        // Add a line in the csv file.
        fputcsv($handle, $data, ';', '"');
    }

    fclose($handle);
});

$response->headers->set('Content-Type', 'application/force-download');
$response->headers->set('Content-Disposition', 'attachment; filename="bookings.csv"');

return $response;

And as far as I can see there is no code that accumulates data in memory while streaming. I use Symfony 2.7, Doctrine MongoDB ODM 1.0.3 and PHP 5.5.9 with a memory_limit of 256M.


Solution

  • As described in the comment, the problem is related to a not idratable object on the result from the cursor. For catching the exception and write the correct data, you can catch the excection. The loop need to be rewrited in order to catch the exception. As Example, you can do a do while loop as follow:

    do {
            try 
            {
                // this could rise an exception
                $result = $cursor->getNext()
                // Transform the Booking object to a flat array.
                $data = $this->constructRow($result);
    
                // Print CSV header
                if (!$headerPrinted) {
                    fputcsv($handle, array_keys($data), ';', '"');
                    $headerPrinted = true;
                }
    
                // Add a line in the csv file.
                fputcsv($handle, $data, ';', '"');
            } catch (\Exception $e) 
            {
              // do nothing and process next element
            } 
     }
    while($cursor->hasNext())
    

    I would suggest also to use a library for managing the CSV writing, such as:

    hope this help