I have been running a Glassfish4 server with an SSL Certificate for my Websocket based app for a number of months now. I recently switched to Payara5. When I set up the Payara server, everything worked great. Now, the certificate has expired, and after renewing it (same process I've always done), the Payara server still thinks it's using the old certificate.
The process I've been going through since the beginning is:
This works, and has always worked, for my Glassfish server. But not the Payara server. I've had to revert to Glassfish while I try and figure this out. I checked in the keystore in Payara for the alias, and it shows the new certificate's expire date. So, as far as I can tell, it should be fine.
The commands I've used to renew the certificate are as follows:
openssl pkcs12 -export -in $LETS_ENCRYPT/live/www.mywebsite.com/fullchain.pem -inkey $LETS_ENCRYPT/live/www.mywebsite.com/privkey.pem -out $LETS_ENCRYPT/live/www.mywebsite.com/pkcs.p12 -name mywebsite.com
keytool -importkeystore -deststorepass *password* -destkeypass *password* -destkeystore $LETS_ENCRYPT/live/www.mywebsite.com/letsencrypt.jks -srckeystore $LETS_ENCRYPT/live/www.mywebsite.com/pkcs.p12 -srcstoretype PKCS12 -srcstorepass *password* -alias mywebsite.com
keytool -importkeystore -srckeystore $LETS_ENCRYPT/live/www.mywebsite.com/letsencrypt.jks -destkeystore $PAYARA_HOME/glassfish/domains/domain1/config/keystore.jks
$PAYARA_HOME/bin/asadmin restart-domain
I checked the keystore afterwords with:
keytool -v -list -keystore $PAYARA_HOME/glassfish/domains/domain1/config/keystore.jks -alias mywebsite.com
and it shows:
Certificate[1]:
Owner: CN=www.mywebsite.com
Issuer: CN=Let's Encrypt Authority X3....
Serial number: ******
Valid from: Wed Apr 03 17:22:48 PDT 2019 until: Tue Jul 02 17:22:48 PDT 2019
The error from the client app shows:
…
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.glassfish.grizzly.ssl.SSLUtils.sslEngineWrap(SSLUtils.java:427)
at org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:337)
at org.glassfish.grizzly.ssl.SSLUtils.handshakeWrap(SSLUtils.java:303)
at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:673)
at org.glassfish.grizzly.ssl.SSLFilter.doHandshakeStep(SSLFilter.java:308)
at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:598)
at org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:310)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:515)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:966)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:963)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416)
at org.glassfish.grizzly.ssl.SSLUtils.executeDelegatedTask(SSLUtils.java:250)
at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:684)
... 17 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:270)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1501)
... 25 more
Caused by: java.security.cert.CertPathValidatorException: validity check failed
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:223)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
... 31 more
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Fri May 17 12:21:27 PDT 2019
at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:274)
at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629)
at sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190)
at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
... 36 more
Which would indicate it is using the old certificate, or something along those lines.
I discovered that I have two keystore.jks locations. The one that my Payara server defaults to is located in the
$PAYARA_HOME/glassfish/domains/domain1
not
$PAYARA_HOME/glassfish/domains/domain1/config
The JVM Option for the keystore location is set to :
-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks,
so I thought I was altering the correct keystore. But, I suppose Payara doesn't use this value, or I have it set wrong. Either way, I just changed my script file to use the domain1/keystore.jks instead of domain1/config/keystore.jks.
Perhaps I goofed my setup and this all could have been avoided. But I figure I should leave this here anyway, in case someone else has a similar issue.