I have a potential memory leak here. It's probably Kohana's. I hope to get insights from you. (Been trying to debug this script for days, I finally gave up last night).
Here's what this script does:
Basically, this is a Background Worker (hosted at Pagodabox). That's the reason why it's an infinite while loop. Step by step:
Problem: memory is not cleared after the first record and eventually, the allocated memory, however huge, it is runs out. I would have to think that since you're processing a new database object, the memory will have to be cleared. But it's not the case.
I've tried different approaches you can think of the clear the memory. I've tried making variables NULL, unset(), I've declared gc_enable, gc_collect_cycles(), I've tried to make the script sleep by 60 seconds hoping that the garbage collector will do his job during lull moments (wink -- I know I'm crazy). I thought ORM is the issue, so I tried DB. I've tried turning profiling off -- which helped by the way, but isn't the problem first and foremost. I've tried __destruct.
Memory still runs out.
You might say that probably the data sets are huge. It is. But the thing is, this can process about 3 to 4 article records before it runs out of memory. Meaning, it can actually process at least one. And I'm hoping that after 1 record, memory is freed.
public function action_grab_article_topics()
{
gc_enable();
$x = 1;
while ($x == 1)
{
// $articles = ORM::factory('article')
// ->where('sent', '=', 0)
// // ->limit(1)
// ->find();
$articles = DB::select()
->from('articles')
->where('sent', '=', 0)
->as_object()
->execute()
->current();
$topics = array();
//foreach ($articles as $a)
//{
//$topics = $articles->topics->find_all()->as_array();
if ($articles)
{
$topics = DB::select('topic_id')
->from('articles_topics')
->where("article_id", '=', $articles->id);
if (! empty($topics))
{
$members = DB::select('members.id', 'members.email', 'members_topics.topic_id', 'topics.topic_name')
->from('members_topics')
->join('members')
->on('members.id', '=', 'members_topics.member_id')
->join('topics')
->on('topics.id', '=', 'members_topics.topic_id')
->where('members_topics.topic_id', 'IN', $topics)
// ->limit(20)
->as_object()
->execute();
foreach ($members as $m)
{
$topic_id = $m->topic_id;
$topic_name = $m->topic_name;
$data = array(
"member_id" => $m->id,
"topic_id" => $topic_id,
"article_id" => $a->id,
"topic_name" => $topic_name,
"title" => $a->title,
);
$emailersched = ORM::factory('emailersched')->values($data)->save();
unset($m);
}
$members->__destruct();
//sleep(2);
//$m = NULL;
}
$data = array('sent'=> time());
$query = DB::update('articles')
->set($data)
->where('id', '=', $articles->id)
->execute();
// $articles->sent = time();
// $articles->save();
//}
//echo "done";
//$a = NULL;
//$articles->__destruct();
$articles = NULL;
$topics = NULL;
unset($articles);
unset($topics);
}
gc_collect_cycles();
//sleep(60);
}
}
Edit: Further in my 'memory leak' investigation (as I continue to encounter problems with my code), here are some weird stuff I've encoutered:
http://pastebin.com/c7pc5XjW This code is run on both Kohana and FuelPHP - identical, basic codebase, uses built in DB module, no ORM used, accessed the same Database. Fuel did not use oil. It was accessed through the public http the way Kohana was accessed.
This code tries to process 50 records of about 10k records all in all.
Here's my memory log for Kohana: http://pastebin.com/gUiF9D2w
Here's my memory log for FuelPHP: http://pastebin.com/v8Pzwu77
Notice that Kohana started at 3Mb and ended at 7Mb. FuelPHP, however, started with around 11Mb, but ended with 11MB too. While Kohana started small, it looked to me that it does have a leak in this regard.
Any thoughts?
Here's a juicy framework war for you: in my desperation, I tried to run the same code to FuelPHP (via Tasks on OIL).
Result:
-302k records processed in 2 minutes -memory leak GONE!
So, I guess the conclusion here is that there's a leak on Kohana codebase somewhere, somehow.