Search code examples
phplaravelcsvcomposer-phpflysystem

League CSV package - reading one line at a time from a resource/stream


I'm using The PHP League CSV importer/exporter to import a large CSV file in Laravel. Since the file is large, I would like to stream it to the CSV parser and handle it one line at a time, without loading every line into memory.

Laravel uses flysystem for the underlying filesystem, and I am using that to obtain a PHP resource to the source CSV.

What I don't understand is how - if it is at all possible - I can feed that resource stream into League CSV so that it reads one line at a time for me to process, before reading in the next line. All the documentation seems to imply that a CSV file is always read fully into memory, and that is what I want to avoid.

Do I need to use callbacks? If so, how can I be sure the stream resource is only being read one line at a time as needed, and not all at once?

I'm guessing I start by creating a stream reader?

use League\Csv\Reader;
$reader = Reader::createFromStream($resource, 'r');

Solution

  • You can iterate over the rows without loading the whole file by using the IteratorAggregate interface of the Reader. So you basically just do

    foreach ($reader as $row) {
        // do stuff
    }
    

    If you are using a mac to read or create the CSV file you will need to add this to your code for it to work correctly:

    if (!ini_get("auto_detect_line_endings")) {
        ini_set("auto_detect_line_endings", '1');
    }