Search code examples
javasslnginxclient-certificates

Java - Ssl stream is cancelled when using client certificate authentication


I cannot figure this one out.

I have a unit test that performs a connection to my service using a client certificate authentication.

// generate a valid client cert and store it in a keystore
String keystorePassword = "xxx";
InputStream pkcs12 = UnitTests.generatePkcs12ForUser(user, keystorePassword, 3600);
KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(pkcs12, keystorePassword.toCharArray());


String url = getBaseServerUrl();

// prepare a ssl context that has the keystore with client cert and key
SSLContext sslContext = SSLContexts.custom()
                                   .loadKeyMaterial(ks, keystorePassword.toCharArray())
                                   // trust all SSL certs
                                   .loadTrustMaterial((X509Certificate[] chain, String authType) -> true)
                                   .build();

// validate any hostname, and don't follow 3XX responses
HttpClient httpClient = HttpClients.custom()
                                   .setSSLContext(sslContext)
                                   .setSSLHostnameVerifier((a,b) -> true)
                                   .disableRedirectHandling()
                                   .build();

// this fails catastrophically
HttpResponse response = httpClient.execute(new HttpGet(url));

I'm using Java 8 and my server is behind a reverse proxy that uses Nginx.

My unit test fails with an exception like this:

javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
[...]
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.InputRecord.read(InputRecord.java:505)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
    ... 43 more

And I can see in the Nginx error.log the following line:

2018/05/18 15:34:12 [crit] 42#42: *327 SSL_do_handshake() failed (SSL: error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error) while SSL handshaking, client: 172.18.0.1, server: 0.0.0.0:443

I have SCOURGED the internet to find the reason for this error, and I feel like I have exhausted all recourse.

Non exhaustive list of things I have tried:

  • Force java to use TlsV1.1 with -Dhttps.protocols=Tlsv1.1 -> still fails
  • Tried to specify the "key strategy" for the loadTrustMaterial to always use my own key -> still fails
  • Used a Jersey client with the same set of Ssl /Keystore params -> still fails
  • Tried all JDK versions of 1.8: OpenJDK, Oracle, and Oracle with Crypto Extensions (JCE)
  • Searching the nginx error seems to imply that the passed cert wasn't correct, so I...
  • ...dumped the key and cert into a couple of PEM files while in debugger and curled the same address -> it works ?!
  • Wiresharked the connection and examined the TLS negotiation. Compared it with a working sample (the simple curl -k <url> above which works flawlessly):
    • Saw something about ALPN but enabling it in Java doesn't fix the exception.
    • Saw different crypto algorithms announced but nothing stands out

Do you guys have any idea? I'm starting to get crazy. I have a hunch that I'm not setting my connection correctly, but I can't see where.


Solution

  • For those that didn't read the comments: I figured it out: I was using the subjectAltName as a storage for something that isn't an alternative name.

    It seems that Java is not happy with this and refuses the connection.