Search code examples
javasocketssslrmi

SSL handshake error in a JavaRMI application

I'm having some trouble with the very beginning of SSL handshake in a JavaRMI application.

My application works well with the standard RMI sockets but it doesn't complete the handshake if I change the default sockets to SSL sockets following this guide: https://blogs.oracle.com/lmalventosa/entry/using_the_ssl_tls_based

So, I changed the constructor of my Activatable Server from

public MyActivatableServer(ActivationID id, MarshalledObject<MyType> obj)
        throws RemoteException
{
    // Some code ...
    // int port = ...
    // Other code ...

    Activatable.exportObject(this, id, port);
}

to

public MyActivatableServer(ActivationID id, MarshalledObject<MyType> obj)
        throws RemoteException
{
    // Some code ...
    // int port = ...
    // Other code ...

    Activatable.exportObject(this, id, port, new SslRMIClientSocketFactory(),
        new SslRMIServerSocketFactory());
}

Then I created a Keystore and a Truststore following this guide: http://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#CreateKeystore

At this point on the server-side I have these files:

  • keystore ( with password: "password" );
  • truststore ( with password: "trustword" );
  • user.cer ( the self-signed certificate ).

And I copied trustore and user.cer on client-side.

Finally I added these properties to the JVM which launch the Activatable Server:

  • -Djavax.net.ssl.keyStore=keystore
  • -Djavax.net.ssl.keyStorePassword=password

and on the Client JVM I added:

  • -Djavax.net.ssl.trustStore=truststore
  • -Djavax.net.ssl.trustStorePassword=trustword
  • -Djavax.net.debug=ssl

This is the complet output of the ssl debug: http://pastebin.com/sxbLhTnF

I can't understand why the client receives an Alert after the "ClientHello":

*** ClientHello, TLSv1
RandomCookie:  GMT: 1410959810 bytes = { 20, 172, 181, 158, 246, 172, 183, 30, 232, 42, 80, 36, 77, 5, 67, 56, 30, 191, 170, 142, 14, 2, 113, 241, 183, 154, 213, 24 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
***
main, WRITE: TLSv1 Handshake, length = 149
main, READ: TLSv1 Alert, length = 2
main, RECV TLSv1 ALERT:  fatal, handshake_failure
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
Solution

  • Finally I added these properties to the JVM which launch the Activatable Server:

    -Djavax.net.ssl.keyStore=keystore
-Djavax.net.ssl.keyStorePassword=password

    You need to define those properties in the ActivationGroupDesc in your setup program.