Search code examples
javatomcatout-of-memoryjmap

Java jmap -heap command gave OutOfMemoryError

We are periodically runnign jmap -heap command to monitor a tomcat application. However this is first time when we got OutOfMemoryError.

When we monitor the memory at server level there is around 110MB available. Since it is just monitoring can we ignore this time assuming no harm has been done to the application and the server was low on memory when this command run?

$ ./jmap -heap 13511
Attaching to process ID 13511, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jmap.JMap.runTool(JMap.java:179)
        at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.<init>(LinuxDebuggerLocal.java:210)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebuggerLinux(BugSpotAgent.java:816)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebugger(BugSpotAgent.java:518)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:493)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
        at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:39)

UPDATE: We tried to run jmap -heap again and got java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.

So, the problem seems to be at OS resource level.

$ ./jmap -heap 13511
#
# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
#
#  Internal Error (gcTaskThread.cpp:38), pid=13162, tid=1104300352
#  Error: Cannot create GC thread. Out of system resources.
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode linux-amd64 )
# An error report file with more information is saved as:
# /data01/home/s618199/JAVA/jdk1.6.0_18/bin/hs_err_pid13162.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
Aborted
Solution

  • This is not a memory problem (even though it is reported as one) but an operating system resource problem: You can not create any more threads. And it seems to affect the jmap program and not Tomcat.