expat parser: memory consumption

Question

I am using expat parser to parse an XML file of around 15 GB . The problem is it throws an "Out of Memory" error and the program aborts .

I want to know has any body faced a similar issue with the expat parser or is it a known bug and has been rectified in later versions ?

Answer 1

I've used expat to parse large files before and never had any problems. I'm assuming you're using SAX and not one of the expat DOM wrappers. If you are using DOM, then that's your problem right there - it would be essentially trying to load the whole file into memory.

Are you allocating objects as you parse the XML and maybe not deallocating them? That would be the first thing I would check for. One way to check if the problem is really with expat or not - if you reduce the program to a simple version that has empty tag handlers (i.e. it just parses the file and does nothing with the results) does it still run out of memory?