In my process I continuously create a new Thread object (subclass of Thread actually) (up to several per second), run it and cleanly end.
I have noticed that when the process has been up for 25 days for instance, the process can die leaving a hprof behind, so that means an OOM. But the heap dump is tiny compared to allocated memory to the heap, so it probably is a PermGen OOM, and I am trying to find out the culprit.
I am not using any special jvm param barring -XX:+HeapDumpOnOutOfMemoryError
I'm answering this in order to bring closure. After much investigation, at least in my case, creation of very large number of thread was not the culprit of the OOM.
I ruled it out in two ways:
the issue was reported in another couple of instances and not many threads had been spawned in those cases.
I created a test where I spawned over 2.5 million threads (doing the normal work they do in our case) and got no issue of OOM.
So, the mere very large number of thread created is not an issue.