I am trying to connect a java client with a JMX server using SSL certificate.
But unfortunately this connection gives a SSL Handshake error. When I tried to debug this, using -Djavax.net.debug=all flag, I get the following error message on the java client.
2016-07-15T13:29:50.02-0700 [APP/0] OUT RMI RenewClean-[10.200.0.27:44445,javax.rmi.ssl.SslRMIClientSocketFactory@305a99f7], READ: TLSv1.2 Alert, length = 2
2016-07-15T13:29:50.02-0700 [APP/0] OUT RMI RenewClean-[10.200.0.27:44445,javax.rmi.ssl.SslRMIClientSocketFactory@305a99f7], RECV TLSv1 ALERT: fatal, handshake_failure
2016-07-15T13:29:50.02-0700 [APP/0] OUT RMI RenewClean-[10.200.0.27:44445,javax.rmi.ssl.SslRMIClientSocketFactory@305a99f7], called closeSocket()
2016-07-15T13:29:50.02-0700 [APP/0] OUT RMI RenewClean-[10.200.0.27:44445,javax.rmi.ssl.SslRMIClientSocketFactory@305a99f7], handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
When I change the client to something else, like a JConsole, the connection works fine using the same SSL certificate and the truststore, which indicates that there is nothing wrong with the SSL certificate or the domain its connecting to.
It seems that the handshake was failing due to incorrect TLS version types.
My client was running on Java7 which was using TLS version 1, while the server was running Java8 which was using TLS version 1.2.
Check this oracle blog, which mentions that Java8 will default use TLS version 1.2
So, when I upgraded my client to use Java8, the issue got resolved.
Hope, this helps someone else as well.