I am running an Apache Storm topology on a server using a Java 7 JVM. I've been considering some JVM tuning and noticed it is currently using a Concurrent Mark and Sweep (CMS) garbage collector. This makes sense, as the server has 32 cores and while it is running multiple JVMs with this setting, it is only running 4 such JVMs which is less than 32 cores.
However, I noticed we're running the garbage collector with the setting CMSConcurrentMTEnabled turned off. The default is for that setting to be turned on, which makes me wonder why anyone would choose to use a single thread for concurrent garbage collection when other threads are available. Under what conditions does using that setting make sense, supposing other threads are available?
(Editing for more detail on my current situation, with hopes that it will lead to an answer) :
The JVMs seem to be running out of memory, performing minor GCs repeatedly that are taking ~9 seconds each, and eventually crashing. No OutOfMemoryError has been thrown, which baffles me. The minor GCs are cleaning up a few kB each time and the overall usage hovers around 100% for quite a while. The heap size is 4 GB so those conditions should trigger an OutOfMemoryError. However, this is getting into a very localized situation for an SO question, I fear.
So, according to experts on Twitter and some mailing lists, there is a bug in Java 6 update 21 where
...CMS won't always free objects with finalizers[.]
The fix is to disable the setting CMSConcurrentMTEnabled. (Or use a newer version of Java.) I am not certain which version of Java was the first to fix this issue.