after we have migrated to PHP 5.5.6
and Apache 2.4
from PHP 5.3.3
every site that runs on Kohana 3.3
encounters from time to time a Out of memory
exception.
Full error message
PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 140316914598360 bytes) in Unknown on line 0
PHP says it tries to allocate 127TB
of memory (bytes at the end change, but its still around TB). It happens totally randomly. Sometimes you get a streak for few hours without a problem and the all of a sudden 1 or even 10 out of memory messages in row. It happens across all projects, be it webservices connecting to MySQL, MSSQL or MongoDB or plain website.
Worst thing is this - Unknown on line 0
which tells me absolutely nothing. Even after disabling the registered shutdown_function
, the message did not change.
At first I have suspected PDO dblib + freetds
as the culprit, but got nowhere using while($row = $query->fetch())
as described in https://bugs.php.net/bug.php?id=64511 or this question.
So far I have tried:
They all show that the memory allocation is in perfect condition.
Only strace
shows some promise, since before every out of memory
exception thrown, is a bunch of munmap
calls, which shows the memory keeps decreasing and then suddenly, throws an exception. Although I am not sure its reliable since the logs seemed mixed at times (from different calls).
Has anyone come across something similar?
What are the best steps to debug this problem?
At the moment, I am out of ideas in what I can do to resolve this...
I am going to answer this one for myself and for future reference.
The actual bug was in pdo_dblib
after all. According to this bug - https://bugs.php.net/bug.php?id=67130 which I found yesterday PDOStatement::nextRowset()
caused memory corruption and thus all our problems. I verified it by removing this portion of our code and let it run without it for 12 hours in "rush" traffic.
This bug was fixed along with https://bugs.php.net/bug.php?id=64511 and I did not notice at first that those two are related.
Our system runs on CentOS and PHP is installed from Software Collections, so we downloaded the source code of said module and applied the patch manually since two versions of the same package from Remi's RPM repository did not work or weren't compatible.
It's really funny, because we migrated from 5.3.3
to 5.5.6
because of https://bugs.php.net/bug.php?id=57593 (there are several bug reports and feature requests) where PDOStatement::nextRowset()
was not implemented (it was fixed in 5.3.7).