Search code examples
javajasper-reportsout-of-memoryexport-to-csv

Jasper Reports OutOfMemoryError on export

I have written a web app for managing and running Jasper reports. Lately I've been working with some reports that generate extremely large (1500+ page) outputs, and attempting to resolve the resultant memory issues. I have discovered the JRFileVirtualizer, which has allowed me to run the report successfully with a very limited memory footprint. However, one of the features of my application is that it stores output files from previously run reports, and allows them to be exported to various formats (PDF, CSV, etc.). Therefore, I find myself in the situation of having a 500+MB .jrprint file and wanting to export it to, for example, CSV on demand. Here is some simplified example code:

JRCsvExporter exporter = new JRCsvExporter();
exporter.setParameter(JRExporterParameter.INPUT_FILE_NAME, jrprintPath);
exporter.setParameter(JRExporterParameter.OUTPUT_STREAM, outputStream);
exporter.exportReport();

Unfortunately, when I attempt this on the large file I mentioned, I get an OutOfMemoryError:

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.io.ObjectInputStream$HandleTable.grow(ObjectInputStream.java:3421)
    at java.io.ObjectInputStream$HandleTable.assign(ObjectInputStream.java:3227)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1744)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
    at java.util.ArrayList.readObject(ArrayList.java:593)
    at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
    at net.sf.jasperreports.engine.base.JRVirtualPrintPage.readObject(JRVirtualPrintPage.java:423)
    ...

From browsing some of the Jasper internals, it looks like no matter how I attempt to set up this export (I have also tried loading and setting the JASPER_PRINT parameter directly), there will ultimately be a call to JRLoader.loadObject(...) which will attempt to load my entire 500MB report into memory (see net.sf.jasperreports.engine.JRAbstractExporter.setInput()).

My question is, is there a way around this that doesn't involve just throwing memory at the problem? 500MB is doable, but it doesn't leave my application very future-proof, and the JRVirtualizer solution for report execution leaves me hoping there will be something similar for export. I am willing to get my hands dirty and extend some of the Jasper internal classes, but the ideal solution would be one provided by Jasper itself, for obvious reasons.

Solution

  • Since posting this question, I have also filed a feature request with JasperSoft. As a follow-up, I was pointed to the JRVirtualizationHelper.setThreadVirtualizer method. This method allows you to set a JRVirtualizer associated with the current thread, which will be used during JasperPrint deserialization.

    I have tested this in my project with satisfactory results. It seems the feature I was hoping existed does indeed exist, although its visibility in the API could potentially be improved.

    Code sample:

    JRVirtualizer virtualizer = new JRSwapFileVirtualizer(1000, new JRSwapFile(reportFilePath, 2048, 1024), true);
JRVirtualizationHelper.setThreadVirtualizer(virtualizer);