Search code examples
phpsql-serverbulk-load

PHP sqlsrv BULK INSERT incomplete


I am trying to insert a big file (few millions row) via SQL server BULK INSERT functionality. My SQL query will look like:

 BULK INSERT MY_TABLE
 FROM '\\myserver\open\myfile.csv'
 WITH (
 firstrow=2,
 FIELDTERMINATOR = ',',
 ROWTERMINATOR = '\n',
 BATCHSIZE = 50000,
 ERRORFILE = '\\myserver\open\myfileerror.log' 
 );

When I trigger it from MSSQL Server Management Studio, it always import it completely. When I do it from my PHP code, sometimes it stop in the middle, without any error messages.

I tried with both sqlsrv_query or sqlsrv_prepare/sqlsrv_execute, same result.

sql; //like the query above
$statement = sqlsrv_query($connection, $sql);
if($statement === false) {
    $error = sqlsrv_errors();
    $error['sql'] = $sql;
    throw new Exception(json_encode($error));
}

Would it be possible to get the logs of MSSQL from the $statement, the same I get from the MSSQL Studio? e.g. (50000 row(s) affected).

As a workaround, I have increased the BATCHSIZE to 1000000, but that is not a real solution.

Background information: - PHP 7.1.9 - sqlsrv version: 4.3.0+9904 - sqlsrv.ClientBufferMaxKBSize: 10240 - Windows 2012 R2 Server


Solution

  • The issue was about the statement buffer. When I read it with sqlsrv_next_result, processing continue.

    $statement = sqlsrv_query($connection, $sql);
    if($statement === false) {
        $error = sqlsrv_errors();
        $error['sql'] = $sql;
        throw new Exception(json_encode($error));
    } else if($statement) {
        while($next_result = sqlsrv_next_result($statement)){
            #echo date("Y-m-d H:i:s",time()). " Reading buffer...\n";
        }
    }