Search code examples
javac++java-native-interfacewebspheregeronimo

Access violation with JNI crash server JVM


I use native dll library for cryptography in IBM Websphere Common Edition 3 JVM 1.7. I send files for checking signature. When I try to test this possibility it works fine. But sometimes there is an error, which crash server JVM. So there are two problems:

  1. Why the error is happens? (I already start to analyze log but not already finished, I hope that somebody meet that problem or have ideas).
  2. What can I do to isolate server from dll crashes? It's very critical that errors in dll don't crash all server. I read that on of the solution is run native code in another JVM. How can I realize this in my particular case? Is that good idea to use ProcessBuilder under WASCE EJB Container?

The error message:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000007fefcd77376, pid=13408, tid=12700
#
# JRE version: 7.0_25-b17
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [CRYPT32.dll+0x7376]  CertFreeCTLContext+0x46
#
# Core dump written. Default location: C:\IBM\WebSphere\AppServerCommunityEdition\shortcuts\hs_err_pid13408.mdmp
#
# An error report file with more information is saved as:
# C:\IBM\WebSphere\AppServerCommunityEdition\shortcuts\hs_err_pid13408.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


Stack: [0x0000000022260000,0x0000000022360000],  sp=0x000000002235ceb0,  free space=1011k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [CRYPT32.dll+0x7376]  CertFreeCTLContext+0x46
j  cryptopro.PKCS7Signature.verifyPKCS7Attached(Lcryptopro/ICertificate;[B)V+16
j  com.it.projects.crypto.CryptoHelp.verify([B[BLjava/security/cert/X509Certificate;I)Z+135
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/VerifyFileData;)Lcom/it/projects/ozi/agent/commands/model/VerifyFileResult;+70
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+5
j  com.it.projects.ozi.agent.commands.Command.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+20
j  com.it.projects.ozi.ejb.OziCommandMessages.processCommand(Lcom/it/projects/ozi/commands/parsers/ICommandParser;Lcom/it/projects/ozi/ejb/pojo/WaitingFiles;)Ljava/util/Properties;+159
j  sun.reflect.GeneratedMethodAccessor403.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+48

I don't call method CertFreeCTLContext externally maybe It's internall error.

UPDATE: full log:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000007fefcd77376, pid=15636, tid=12260
#
# JRE version: 7.0_25-b17
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [CRYPT32.dll+0x7376]  CertFreeCTLContext+0x46
#
# Core dump written. Default location: C:\IBM\WebSphere\AppServerCommunityEdition\shortcuts\hs_err_pid15636.mdmp
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x000000001832a000):  JavaThread "ConnectorThreadPool 103" daemon [_thread_in_native, id=12260, stack(0x000000003dec0000,0x000000003dfc0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x000000000000033c

Registers:
RAX=0x0000000037994c00, RBX=0x0000000037994bb0, RCX=0x0000000037994bb0, RDX=0x0000000000000002
RSP=0x000000003dfbbde0, RBP=0x000000003dfbc320, RSI=0x000000003dfbc178, RDI=0x000000000000032c
R8 =0x0000000000000001, R9 =0x00000000000003c2, R10=0x0000000000000261, R11=0x00000000004a0158
R12=0x0000000000000000, R13=0x00000006e9d8d000, R14=0x000000003dfbc350, R15=0x000000001832a000
RIP=0x000007fefcd77376, EFLAGS=0x0000000000010202

Top of Stack: (sp=0x000000003dfbbde0)
0x000000003dfbbde0:   000000003dfbbfe8 000000003dfbc178
0x000000003dfbbdf0:   000000003dfbc320 000007fefcfe1512
0x000000003dfbbe00:   000000003dfbbfe8 000007fefcd77347
0x000000003dfbbe10:   00000006e9d8d000 0000000000000026
0x000000003dfbbe20:   0000000100000001 000000003dfbc178
0x000000003dfbbe30:   000000003dfbbfe8 000007feeffe5d9d
0x000000003dfbbe40:   00000000379d6c20 00000006e9d8d000
0x000000003dfbbe50:   000000003dfbc178 0000000000000000
0x000000003dfbbe60:   000000003dfbbfa4 000000003dfbbf24
0x000000003dfbbe70:   000000003dfbbf44 000000003dfbbf64
0x000000003dfbbe80:   000000003dfbbea8 000000003dfbbec8
0x000000003dfbbe90:   0000000000000000 cccccccccccccccc
0x000000003dfbbea0:   cccccccccccccccc 00000000379aee70
0x000000003dfbbeb0:   cccccccccccccccc cccccccccccccccc
0x000000003dfbbec0:   cccccccccccccccc 00000000379b5fc0
0x000000003dfbbed0:   cccccccccccccccc cccccccccccccccc 

Instructions: (pc=0x000007fefcd77376)
0x000007fefcd77356:   c3 90 90 90 90 90 90 90 90 90 48 85 c9 74 f1 48
0x000007fefcd77366:   89 5c 24 08 57 48 83 ec 20 48 8b 79 18 48 8b d9
0x000007fefcd77376:   8b 4f 10 f0 83 43 0c ff 0f 84 bf 06 00 00 f6 c1
0x000007fefcd77386:   04 0f 85 21 07 00 00 48 8b 5c 24 30 eb bd 0f ba 


Register to memory mapping:

RAX=0x0000000037994c00 is an unknown value
RBX=0x0000000037994bb0 is an unknown value
RCX=0x0000000037994bb0 is an unknown value
RDX=0x0000000000000002 is an unknown value
RSP=0x000000003dfbbde0 is pointing into the stack for thread: 0x000000001832a000
RBP=0x000000003dfbc320 is pointing into the stack for thread: 0x000000001832a000
RSI=0x000000003dfbc178 is pointing into the stack for thread: 0x000000001832a000
RDI=0x000000000000032c is an unknown value
R8 =0x0000000000000001 is an unknown value
R9 =0x00000000000003c2 is an unknown value
R10=0x0000000000000261 is an unknown value
R11=0x00000000004a0158 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x00000006e9d8d000 is an oop
{method} 
 - klass: {other class}
R14=0x000000003dfbc350 is pointing into the stack for thread: 0x000000001832a000
R15=0x000000001832a000 is a thread


Stack: [0x000000003dec0000,0x000000003dfc0000],  sp=0x000000003dfbbde0,  free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [CRYPT32.dll+0x7376]  CertFreeCTLContext+0x46
j  cryptopro.PKCS7Signature.verifyPKCS7Attached(Lcryptopro/ICertificate;[B)V+16
j  com.it.projects.crypto.CryptoHelp.verify([B[BLjava/security/cert/X509Certificate;I)Z+135
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/VerifyFileData;)Lcom/it/projects/ozi/agent/commands/model/VerifyFileResult;+70
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+5
j  com.it.projects.ozi.agent.commands.Command.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+20
j  com.it.projects.ozi.ejb.OziCommandMessages.processCommand(Lcom/it/projects/ozi/commands/parsers/ICommandParser;Lcom/it/projects/ozi/ejb/pojo/WaitingFiles;)Ljava/util/Properties;+159
v  ~StubRoutines::call_stub

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  cryptopro.nativeCertificateUtilities.verifyPKCS7AttachedSignature([B[B)Lcryptopro/PKCS7VerifyingResult;+0
j  cryptopro.PKCS7Signature.verifyPKCS7Attached(Lcryptopro/ICertificate;[B)V+16
j  com.it.projects.crypto.CryptoHelp.verify([B[BLjava/security/cert/X509Certificate;I)Z+135
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/VerifyFileData;)Lcom/it/projects/ozi/agent/commands/model/VerifyFileResult;+70
j  com.it.projects.ozi.agent.commands.executors.VerifyFileExecutor.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+5
j  com.it.projects.ozi.agent.commands.Command.execute(Lcom/it/projects/ozi/agent/commands/model/AbstractCommandData;)Lcom/it/projects/ozi/agent/commands/model/AbstractCommandResult;+20
j  com.it.projects.ozi.ejb.OziCommandMessages.processCommand(Lcom/it/projects/ozi/commands/parsers/ICommandParser;Lcom/it/projects/ozi/ejb/pojo/WaitingFiles;)Ljava/util/Properties;+159
v  ~StubRoutines::call_stub
j  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
j  sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.monitoring.StatsInterceptor.record(Ljavax/interceptor/InvocationContext;Ljava/lang/reflect/Method;)Ljava/lang/Object;+32
j  org.apache.openejb.monitoring.StatsInterceptor.invoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+3
j  sun.reflect.GeneratedMethodAccessor907.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.cdi.CdiInterceptor.invoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+185
j  org.apache.openejb.cdi.CdiInterceptor.access$000(Lorg/apache/openejb/cdi/CdiInterceptor;Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+2
j  org.apache.openejb.cdi.CdiInterceptor$1.call()Ljava/lang/Object;+8
j  org.apache.openejb.cdi.CdiInterceptor.aroundInvoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+11
j  sun.reflect.GeneratedMethodAccessor906.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.core.interceptor.InterceptorStack.invoke([Ljava/lang/Object;)Ljava/lang/Object;+24
j  org.apache.openejb.core.stateless.StatelessContainer._invoke(Ljava/lang/reflect/Method;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/apache/openejb/core/stateless/Instance;Lorg/apache/openejb/core/ThreadContext;Lorg/apache/openejb/InterfaceType;)Ljava/lang/Object;+108
j  org.apache.openejb.core.stateless.StatelessContainer.invoke(Ljava/lang/Object;Lorg/apache/openejb/InterfaceType;Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+426
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+19
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+31
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(Ljava/lang/Object;Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+166
j  org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+440
j  com.it.projects.ozi.ejb.OziCommandMessages$LocalBeanProxy.processCommand(Lcom/it/projects/ozi/commands/parsers/ICommandParser;Lcom/it/projects/ozi/ejb/pojo/WaitingFiles;)Ljava/util/Properties;+40
j  com.it.projects.ozi.ejb.EhdOziReceiverMessageCallImpl.executeCommand(Ljava/lang/String;)V+202
j  com.it.projects.ozi.ejb.EhdOziReceiverMessageCallImpl.reciveFile(Ljava/io/File;Ljava/lang/String;Ljava/util/Map;)V+235
j  com.it.mqtf.api.ReciveMessageImpl.onMessage(Ljavax/jms/Message;)V+467
j  sun.reflect.GeneratedMethodAccessor981.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.monitoring.StatsInterceptor.record(Ljavax/interceptor/InvocationContext;Ljava/lang/reflect/Method;)Ljava/lang/Object;+32
j  org.apache.openejb.monitoring.StatsInterceptor.invoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+3
j  sun.reflect.GeneratedMethodAccessor907.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.cdi.CdiInterceptor.invoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+185
j  org.apache.openejb.cdi.CdiInterceptor.access$000(Lorg/apache/openejb/cdi/CdiInterceptor;Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+2
j  org.apache.openejb.cdi.CdiInterceptor$1.call()Ljava/lang/Object;+8
j  org.apache.openejb.cdi.CdiInterceptor.aroundInvoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+11
j  sun.reflect.GeneratedMethodAccessor906.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.core.interceptor.InterceptorStack.invoke([Ljava/lang/Object;)Ljava/lang/Object;+24
j  org.apache.openejb.core.stateless.StatelessContainer._invoke(Ljava/lang/reflect/Method;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/apache/openejb/core/stateless/Instance;Lorg/apache/openejb/core/ThreadContext;Lorg/apache/openejb/InterfaceType;)Ljava/lang/Object;+108
j  org.apache.openejb.core.stateless.StatelessContainer.invoke(Ljava/lang/Object;Lorg/apache/openejb/InterfaceType;Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+426
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+19
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+31
j  org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(Ljava/lang/Object;Ljava/lang/Class;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+166
j  org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+440
j  com.sun.proxy.$Proxy214.onMessage(Ljavax/jms/Message;)V+16
j  com.it.projects.ozi.ejb.OziEhdListener.onMessage(Ljavax/jms/Message;)V+27
j  sun.reflect.GeneratedMethodAccessor980.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.monitoring.StatsInterceptor.record(Ljavax/interceptor/InvocationContext;Ljava/lang/reflect/Method;)Ljava/lang/Object;+32
j  org.apache.openejb.monitoring.StatsInterceptor.invoke(Ljavax/interceptor/InvocationContext;)Ljava/lang/Object;+3
j  sun.reflect.GeneratedMethodAccessor907.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+40
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke()Ljava/lang/Object;
J  org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed()Ljava/lang/Object;
j  org.apache.openejb.core.interceptor.InterceptorStack.invoke([Ljava/lang/Object;)Ljava/lang/Object;+24
j  org.apache.openejb.core.mdb.MdbContainer._invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/apache/openejb/BeanContext;Lorg/apache/openejb/InterfaceType;Lorg/apache/openejb/core/mdb/MdbContainer$MdbCallContext;)Ljava/lang/Object;+54
j  org.apache.openejb.core.mdb.MdbContainer.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;Lorg/apache/openejb/InterfaceType;[Ljava/lang/Object;)Ljava/lang/Object;+211
j  org.apache.openejb.core.mdb.EndpointHandler.deliverMessage(Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+136
j  org.apache.openejb.core.mdb.EndpointHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+209
j  com.sun.proxy.$Proxy146.onMessage(Ljavax/jms/Message;)V+16
j  com.ibm.mq.connector.inbound.MessageEndpointWrapper.onMessage(Ljavax/jms/Message;)V+59
j  com.ibm.mq.jms.MQSession$FacadeMessageListener.onMessage(Ljavax/jms/Message;)V+32
j  com.ibm.msg.client.jms.internal.JmsSessionImpl.run()V+168
j  com.ibm.mq.jms.MQSession.run()V+18
j  com.ibm.mq.connector.inbound.ASFWorkImpl.doDelivery(Ljavax/resource/spi/endpoint/MessageEndpoint;)V+85
j  com.ibm.mq.connector.inbound.AbstractWorkImpl.run()V+438
J  org.apache.geronimo.connector.work.WorkerContext.run()V
j  org.apache.geronimo.connector.work.pool.NamedRunnable.run()V+4
j  org.apache.geronimo.pool.ThreadPool$1.run()V+15
j  org.apache.geronimo.pool.ThreadPool$ContextClassLoaderRunnable.run()V+25
J  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )

Other Threads:
  0x00000000125a2800 VMThread [stack: 0x00000000135f0000,0x00000000136f0000] [id=10636]
  0x0000000013bfe800 WatcherThread [stack: 0x00000000144b0000,0x00000000145b0000] [id=13912]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 PSYoungGen      total 1050496K, used 837563K [0x00000007aaab0000, 0x00000007eff50000, 0x0000000800000000)
  eden space 1041152K, 80% used [0x00000007aaab0000,0x00000007ddc9ec28,0x00000007ea370000)
  from space 9344K, 0% used [0x00000007ea370000,0x00000007ea370000,0x00000007eac90000)
  to   space 47040K, 0% used [0x00000007ed160000,0x00000007ed160000,0x00000007eff50000)
 ParOldGen       total 699072K, used 267097K [0x0000000700000000, 0x000000072aab0000, 0x00000007aaab0000)
  object space 699072K, 38% used [0x0000000700000000,0x00000007104d66c0,0x000000072aab0000)
 PSPermGen       total 207808K, used 184647K [0x00000006e0000000, 0x00000006ecaf0000, 0x0000000700000000)
  object space 207808K, 88% used [0x00000006e0000000,0x00000006eb451c30,0x00000006ecaf0000)

Card table byte_map: [0x0000000005880000,0x0000000006190000] byte_map_base: 0x0000000002180000

Polling page: 0x0000000000350000

Code Cache  [0x0000000002880000, 0x0000000003e30000, 0x0000000005880000)
 total_blobs=6031 nmethods=5187 adapters=793 free_code_cache=27187Kb largest_free_block=27617024

Compilation events (10 events):
Event: 86700.998 Thread 0x0000000013bed000 5776             java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock::unlock (10 bytes)
Event: 86701.004 Thread 0x0000000013bed000 nmethod 5776 0x0000000003e1f450 code [0x0000000003e1f5e0, 0x0000000003e1f9d0]
Event: 87009.322 Thread 0x0000000013bed000 5777   !         org.restlet.engine.io.BioUtils::toString (97 bytes)
Event: 87009.341 Thread 0x0000000013bed000 nmethod 5777 0x0000000003e20510 code [0x0000000003e206c0, 0x0000000003e210c8]
Event: 87022.128 Thread 0x0000000013bd0800 5778   !         bpm.rest.client.GenericClient::encodeArguments (201 bytes)
Event: 87022.153 Thread 0x0000000013bd0800 nmethod 5778 0x0000000003e1d610 code [0x0000000003e1d820, 0x0000000003e1e588]
Event: 87024.970 Thread 0x0000000013bed000 5779             com.ibm.db2.jcc.t4.db::m (347 bytes)
Event: 87025.065 Thread 0x0000000013bed000 nmethod 5779 0x0000000003e23a50 code [0x0000000003e23fa0, 0x0000000003e26610]
Event: 87026.073 Thread 0x0000000013bd0800 5780             org.zkoss.zel.CompositeELResolver::getValue (57 bytes)
Event: 87026.076 Thread 0x0000000013bd0800 nmethod 5780 0x0000000003e1ffd0 code [0x0000000003e20120, 0x0000000003e202d8]

GC Heap History (10 events):

Deoptimization events (10 events):

Internal exceptions (10 events):
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd030970 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd030e08 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd0312a0 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd031738 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd031bd0 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd032068 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd032500 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd032998 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd032e30 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716
Event: 87079.936 Thread 0x000000001832a000 Threw 0x00000007dd0332c8 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jni.cpp:716

Events (10 events):


Dynamic libraries:

VM Arguments:
jvm_args: -Xms1024m -Xmx4096m -XX:MaxPermSize=512m -javaagent:C:\IBM\WebSphere\AppServerCommunityEdition\lib\agent\transformer.jar -Dkaraf.startLocalConsole=true -Dkaraf.startRemoteShell=false -Dorg.apache.geronimo.home.dir=C:\IBM\WebSphere\AppServerCommunityEdition -Dorg.apache.geronimo.server.dir=C:\IBM\WebSphere\AppServerCommunityEdition -Dkaraf.home=C:\IBM\WebSphere\AppServerCommunityEdition -Dkaraf.base=C:\IBM\WebSphere\AppServerCommunityEdition -Djava.util.logging.config.file=C:\IBM\WebSphere\AppServerCommunityEdition\etc\java.util.logging.properties -Djava.endorsed.dirs=C:\IBM\WebSphere\AppServerCommunityEdition\lib\endorsed;C:\Program Files\Java\jre7\lib\endorsed -Djava.ext.dirs=C:\IBM\WebSphere\AppServerCommunityEdition\lib\ext;C:\Program Files\Java\jre7\lib\ext -Djava.io.tmpdir=C:\IBM\WebSphere\AppServerCommunityEdition\var\temp 
java_command: org.apache.geronimo.cli.daemon.DaemonCLI
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=C:\Program Files\Java\jre7
JRE_HOME=C:\Program Files\Java\jre7
CLASSPATH=c:\Program Files (x86)\Java\jdk1.6.0_21\;.;C:\PROGRA~1\IBM\SQLLIB\java\db2java.zip;C:\PROGRA~1\IBM\SQLLIB\java\db2jcc.jar;C:\PROGRA~1\IBM\SQLLIB\java\sqlj.zip;C:\PROGRA~1\IBM\SQLLIB\java\db2jcc_license_cu.jar;C:\PROGRA~1\IBM\SQLLIB\bin;C:\PROGRA~1\IBM\SQLLIB\java\common.jar;C:\IBM\WebSphere MQ\java\lib\com.ibm.mqjms.jar;C:\IBM\WebSphere MQ\java\lib\com.ibm.mq.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\collector-tool-api.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\collector-tool.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\commons-cli.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\geronimo-cli.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\geronimo-hook.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\geronimo-main.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\geronimo-rmi-loader.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\karaf-jaas-boot.jar;C:\IBM\WebSphere\AppServerCommunityEdition\lib\pax-logging-api.jar
PATH=C:\IBM\WebSphere MQ\java\lib;C:\IBM\WebSphere MQ\java\lib64;c:\oracle\product\10.2.0\db_1\bin;c:\program files (x86)\collabnet\subversion client;C:\Windows\system32;C:\Windows;C:\Windows\system32\wbem;C:\Windows\system32\windowspowershell\v1.0\;c:\postgresql\9.1\bin\;c:\program files (x86)\java\jre6\bin;c:\program files\java\jdk1.6.0_25\bin;c:\program files\tortoisesvn\bin;c:\program files\ibm\gsk8\lib64;c:\program files (x86)\ibm\gsk8\lib;C:\PROGRA~1\IBM\SQLLIB\BIN;C:\PROGRA~1\IBM\SQLLIB\FUNCTION;C:\PROGRA~1\IBM\SQLLIB\SAMPLES\REPL;C:\IBM\WebSphere MQ\bin64;C:\IBM\WebSphere MQ\bin;C:\IBM\WebSphere MQ\tools\c\samples\bin
USERNAME=db2admin
OS=Windows_NT
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 15 Stepping 1, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows NT 6.1 , 64 bit Build 7600 

CPU:total 4 (1 cores per cpu, 1 threads per core) family 6 model 15 stepping 1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, tsc, tscinvbit

Memory: 4k page, physical 16776760k(4749956k free), swap 33551624k(21757148k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (23.25-b01) for windows-amd64 JRE (1.7.0_25-b17), built on Jun 21 2013 12:58:32 by "java_re" with unknown MS VC++:1600

time: Tue May 20 17:23:53 2014
elapsed time: 87089 seconds

Solution

  • Why the error is happens? (I already start to analyze log but not already > finished, I hope that somebody meet that problem or have ideas).

    In my case it was multithreading problem. I refactored given JNI library for working in multithreading environment and there were no problems any more.

    What can I do to isolate server from dll crashes? It's very critical that errors in dll don't crash all server. I read that on of the solution > is run native code in another JVM. How can I realize this in my particular case? Is that good idea to use ProcessBuilder under WASCE EJB Container?

    Because of this case and some other cases we decided run security subsystem on separate server.