OutOfMemoryError in Weblogic FileStore

I'm trying to stress my Application into Weblogic11g and I'm sending it many JMS Message on the queues.

But, the FileStore crashes with an OOE around 20K messages and a max size of 647,169 ko

Exception in thread "Thread-13" java.lang.OutOfMemoryError: Java heap space
            at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
            at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
            at weblogic.store.io.file.StoreFile.expand(StoreFile.java:324)
            at weblogic.store.io.file.Heap.reserveSpace(Heap.java:305)
            at weblogic.store.io.file.Heap.multiWrite(Heap.java:438)
            at weblogic.store.io.file.FileStoreIO.flush(FileStoreIO.java:497)
            at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:638)
            at weblogic.store.internal.PersistentStoreImpl$2.run(PersistentStoreImpl.java:383)

And a few line of the log file

Feb 25, 2014 7:53:19 PM CET Warning JTA BEA-110484 The JTA health state has changed from HEALTH_OK to HEALTH_WARN with reason codes: Resource WLStore_MyFS_stores-Node1-file-jms declared unhealthy.

Feb 25, 2014 7:53:19 PM CET Warning JTA BEA-110030 XA resource [WLStore_MyFS_stores-Node1-file-jms] has not responded in the last 120 second(s).

Feb 25, 2014 7:53:19 PM CET Warning JTA BEA-110405 Resource WLStore_MyFS_stores-Node1-file-jms was not assigned to any of these servers: Node1

Feb 25, 2014 7:54:19 PM CET Warning JTA BEA-110486 Transaction BEA1-5DA4B1F8A57C83AEDB1B cannot complete commit processing because resource [WLStore_MyFS_stores-Node1-file-jms] is unavailable. The transaction will be abandoned after 3,420 seconds unless all resources acknowledge the commit decision.

Is it possible to increase the size of this FileStore ?

Solution

When sending messages to WLS it keeps the message + header in memory until the message is consumed. If your rate of production of messages is faster then rate of consumption of messages then you will eventually hit an OOM.

There's couple of things you can do to avoid getting OOM

1) Ensure you have enough consumers for the messages and they are able to consume messages quickly.

2) By default there's a JMS Paging feature which gets triggered when the memory consumption of JMS messages is about 1/3 of the overall heap. You can tune your server to trigger Paging earlier if you want. What paging does is, it leaves the header part of each message in memory and the body moves to the paging file, thereby releasing some memory. For simplistic calculation JMS headers only would consume about 1k of memory. - Note Pending messages will have both header and body in memory.

3) Ofcourse increasing the JVM size for the managed server hosting your JMS server would directly enable you to keep more messages in memory.