Search code examples
phparraysfilearray-filter

Quickest way to do PHP array_filter on very big array or load less data from a file?


Problem

I have a txt file that is almost 2 Mb. It has lines with a time, date, name and a message. I would like to do a regular expression on each line, so I start by reading the file into an array.

$array1 = file('<file_name>');

Between each line in the file is a whiteline that I would like to skip. On stackoverflow I found the following peace of code which seems to be ok.

array_values( array_filter($array1, 'trim'));

My problem is that this takes very long to process.

Questions

  1. Is there a way to skip the whitelines while reading the file, before I make the array, so the array that I start with has way less keys?
  2. If I have to use the array_filter function. Which way is faster and best for memory use?

$array2 = array_values( array_filter($array1));

or

$array2 = array_values( array_filter($array1));
unset($array1);

Solution

  • If you want to optimize memory, you'll have to read each line and process it.

    1. Open file
    2. Start loop
    3. read line
    4. process line
    5. save data processed
    6. End loop

    Use fopen, fread, fclose php functions. You'll find some information here