This issue occurs around once in a day on the production environment. The load is around 20TPS and the AES decryption is called twice for every request on the server. The error is not generated for all requests but It crashes once in a day. Following is the snippet of hs_err_pid file.
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007f26cc5de47e, pid=53978, tid=0x00007f26837f7700 JRE version: Java(TM) SE Runtime Environment (8.0_241-b31) (build 1.8.0_241-b31) Java VM: Java HotSpot(TM) 64-Bit Server VM (25.241-b31 mixed mode linux-amd64 compressed oops) Problematic frame: v ~StubRoutines::arrayof_jbyte_fill
Current thread (0x00007f26d0a03800): JavaThread "http-nio-9094-exec-10" daemon
[_thread_in_Java, id=54617, stack(0x00007f26836f7000,0x00007f26837f8000)]
Stack: [0x00007f26836f7000,0x00007f26837f8000], sp=0x00007f26837f44f0, free space=1013k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
v ~StubRoutines::arrayof_jbyte_fill
J 26458 C2 java.util.Arrays.fill([BB)V (21 bytes) @ 0x00007f26cc737ca2
[0x00007f26cc737c60+0x42]
j com.sun.crypto.provider.CipherCore.fillOutputBuffer([BI[BII[B)I+73
j com.sun.crypto.provider.CipherCore.doFinal([BII)[B+65
j com.sun.crypto.provider.AESCipher.engineDoFinal([BII)[B+7
j javax.crypto.Cipher.doFinal([B)[B+30
j com.hello.genesys.common.AES.decrypt([BLjava/lang/String;)[B+92
Following is the decryption method :
public static byte[] decrypt(byte[] data, String password) throws Exception {
try {
log.info("/// Inside the decrypter method of AES class ////");
if (null == cipher) {
log.info("The Cipher is null and hence forming new object of cipher");
cipher = Cipher.getInstance("AES/ECB/PKCS5PADDING");
}
log.info("Key generation method to be called");
Key k = new SecretKeySpec(password.getBytes(), AES_ENCRYPTION_TYPE);
log.info("Cipher initialization to be done");
cipher.init(Cipher.DECRYPT_MODE, k);
log.info("The byte array value to be returned");
return cipher.doFinal(data);
} catch (Throwable e) {
log.info("Inside Exception of AES decryption method");
e.printStackTrace();
}
return null;
}
This method is called in a different class :
byte[] decrypter = AES.decrypt(decodeToken, _env("tokenSalt"));
It does not throws an exception but crashes with arrayof_jbyte_fill related error. Please help me solve this issue as it occurs only in the production environment but not in a lower environment. Link to the hs_err_pid file for more details: https://drive.google.com/file/d/1WDG4rm7vIw6HDbhZPtXjud7r8TEh4d74/view?usp=sharing
The goof-up was with the cipher object itself. I have now created & initialized a local cipher object instead of the global one and there seems to be no crash for 2 weeks after deployment. So as 2 decryption processes were occurring simultaneously back to back, it was getting crashed during the second decryption request as the object was still being used by the first one. So this is my analysis for this issue