Search code examples
javaapache-commonscircular-buffer

Apache Commons - CircularFifoBuffer BufferOverflowException


Background

I am attempting to use the CircularFifoBuffer class from the Apache Commons library that contains a collection of recent messages received over a WebSocket connection. However, when I reach the size limit of the CircularFifoBuffer in a Linux deployment a BufferOverflowException is being thrown.

I am still fairly new to Java but here's how I'm defining the CircularFifoBuffer instance variable:

private static Buffer<String> recentWebSocketMessages = new CircularFifoBuffer<String>(1000);

And here's the exception:

org.apache.commons.collections15.BufferOverflowException: The buffer cannot hold more than 1000 objects.
 at org.apache.commons.collections15.buffer.BoundedFifoBuffer.add(BoundedFifoBuffer.java:218)
 at org.apache.commons.collections15.buffer.CircularFifoBuffer.add(CircularFifoBuffer.java:94)
 at com.example.SystemMonitor.putRecentWebSocketMessage(SystemMonitor.java:228) 

What's strange to me is this problem is intermittent, when I write an infinite loop:

while(true)
{
    recentWebSocketMessages.add("TESTING");
}

The exception doesn't occur (at least in Windows - I haven't been able to test Linux yet)

Question

So I guess the main issue here is why is this problem intermittent and can I solve this by defining the static variable as such instead? (Even though this tightly couples recentWebSocketMessages with the CircularFifoBuffer implementation of BoundedFifoBuffer)

private static CircularFifoBuffer<String> recentWebSocketMessages = new CircularFifoBuffer<String>(1000);

Edit

Thank you sorifiend and Tom for pointing out the simple fact that this implementation is not synchronized. For those interested, this simple thread example below proves that multi-threading is the issue and multiple threads accessing the buffer simultaneously can/will cause the BufferOverflowException to occur (with my original definition of the buffer):

while(true)
{
    new Thread(new Runnable() 
    {
        @Override
        public void run()
        {
            try 
            {
                recentWebSocketMessages.add("TESTING");
            } 
            catch(BufferOverflowException e) 
            {
                e.printStackTrace();
            }
        }
    }).start();
}

The same code with the following definition of recentWebSocketMessages does not throw the BufferOverflowException since accesses to the buffer are synchronized as mentioned in the API and in sorifiend's answer:

Buffer recentWebSocketMessages = BufferUtils.synchronizedBuffer(new CircularFifoBuffer(1000));

Solution

  • Quick answer: No, changing it like that wont really help.

    The testing message wont cause an issue because items should automatically be removed to add space:

    CircularFifoBuffer is a first in first out buffer with a fixed size that replaces its oldest element if full.

    The removal order of a CircularFifoBuffer is based on the insertion order; elements are removed in the same order in which they were added. The iteration order is the same as the removal order.

    However I believe there is sometimes a problem when using multiple threads for your web sockets when you are at your buffer limit (In your case at 1000).

    If this is your problem then the best way to combat it is to synchronize it like so:

    Buffer recentWebSocketMessages = BufferUtils.synchronizedBuffer(new CircularFifoBuffer(1000));

    Instead of how you do it at the moment:

    Buffer<String> recentWebSocketMessages = new CircularFifoBuffer<String>(1000);


    Copied form API: https://commons.apache.org/proper/commons-collections/javadocs/api-3.2.2/org/apache/commons/collections/buffer/CircularFifoBuffer.html