What is the best way to display a large set of data (PHP)

Question

I have a function that connect to about a 1000+ databases and get the data back and put it in an array.

however, the data is so large and I want to display the results on the browser each time the loop goes through a connection instead of waiting for the whole loop to finish.

or if there is a better way of doing so, please share.

function Get_data()
{
 // loop base on the number of available databases
    for ($i = 0; $i < count($client_databases); $i++) {

        // Getting connection for each database
        $database = $this->connect_clients_db($client_databases[$i]);

       // return an array of data from the database
        $ClientShema = $this->getSomeData($database);

        // put the data in array
         $schemas[]=$ClientShema;
    }


     return $schemas;


}

example of results would be

loop 1 (database_one) this is a data coming from database one

loop 2 (database_two) this is a data coming from database two

Answer 1

You can turn on output buffering and flush the buffer periodically to the browser.

First, you have to send a certain amount of data to the browser:

echo str_repeat(" ", 40000);

Next, you need to start output buffering:

ob_start();

Finally, after you've output what you want to send to the browser, you need to flush the output buffer. I found that I had to call the following three functions to get this to work:

ob_end_flush();
ob_flush();
flush();

So, your code might look like the following:

function Get_data()
{
    echo str_repeat(" ", 40000);

    //loop base on the number of available databases
    for ($i = 0; $i < count($client_databases); $i++) {
        //Start buffering output
        ob_start();

        // Getting connection for each database
        $database = $this->connect_clients_db($client_databases[$i]);

        // return an array of data from the database
        $ClientShema = $this->getSomeData($database);

        // put the data in array
        $schemas[]=$ClientShema;

        //Write some output somewhere here.

        //flush the output buffer here.
        ob_end_flush();
        ob_flush();
        flush();
    }

    return $schemas;
}

---

Here is a block of code you can use to test this technique:

<?php

echo str_repeat(" ", 40000);

for ($i = 0; $i < 10; $i++) {
    ob_start();

    echo "Hello world.<br/>";
    ob_end_flush();
    ob_flush();
    flush();

    usleep(500000);
}