ZF2 unbuffered SQL MySQL select

Question

I have a ZF2 console application where I need to read through eight million records in a MySQL database.

I am trying to do an unbuffered MySQL query but I cannot find the proper way using ether the Zend\Db\TableGateway\TableGateway or more directly using Zend\Db\Adapter\Adapter perhaps I am looking in the wrong place?

I am looking for a proper example of reading through millions of rows using Zend Framework 2.

After a long, long, long sessions with the debugger and reading though the ZF2 code I found that the following gave me the desired behavior.

    // Get a database adapter for the source (defalut) db
    $SourceAdapter = $sm->get('Zend\Db\Adapter\Adapter');

    // Turn off source database buffering.
    $SourceAdapter->getDriver()
             ->getConnection()
             ->getResource()
             ->setAttribute(\PDO::MYSQL_ATTR_USE_BUFFERED_QUERY,false);
        $SourceAdapter->getDriver()
              ->getConnection()
              ->getResource()
              ->setAttribute(\PDO::CURSOR_FWDONLY,true);

Answer 1

Look at Buffered and unbuffered queries

If you're using PDOMySql driver, specify an option to disable buffered query in connection config (doctrine.local.php) :

<?php

return [
    'doctrine' => [
        'connection' => [
            'orm_default' => [
                 'driverClass' => 'Doctrine\DBAL\Driver\PDOMySql\Driver',
                 'params' => [
                     'host'     => 'localhost',
                     'port'     => '3306',
                     'user'     => 'root',
                     'password' => '',
                     'dbname'   => 'dbname',
                     'charset'  => 'utf8',
                     'driverOptions' => [
                         PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false
                     ],
                 ],
             ],
        ],
    ],
];

You can use \Doctrine\DBAL\Connection to fetch the data, for example in your code:

/* @var \Doctrine\DBAL\Connection $connection */
$connection = $this->getServiceLocator()->get('doctrine.connection.orm_default');

$sql = 'SELECT * FROM Records';
$stmt = $connection->query($sql);

while ($row = $stmt->fetch()) {
    echo $row['id'];
}

Doctrine DBAL Connection - Data retrieval