java ssl websphere-liberty open-liberty java-http-client

Mutual TLS connection from OpenLiberty with custom keystore

I'm trying to use configured host-specific outbound SSL configuration in OpenLiberty to make an https connection to a remote system (Apple Pay, in this case, but I don't think relevant).

I know the keystore and password and protocols are correct, because when I do it entirely programmatically, like shown in https://stackoverflow.com/a/63080443/796761, the connection and request fully work.

However, when I remove all the keystore/SSL setup and try to fall back on what I've configured in server.xml, I get the same error as I get when I'm not setting up SSL at all. (Which, in this case is java.io.IOException: HTTP/1.1 header parser received no bytes.)

Relevant server.xml elements:

    <featureManager>
        <feature>webProfile-8.0</feature> <!-- adds ssl-1.0 -->
...
    </featureManager>

...
    <keyStore id="applePayKeyStore" location="c:/keys/MerchID.AZMVDNOW.p12" password="redacted"/>

    <ssl id="applePaySSL" keyStoreRef="applePayKeyStore" clientAuthentication="true" sslProtocol="TLSv1.2">
        <outboundConnection host="apple-pay-gateway-cert.apple.com"/>
    </ssl>

Java code:

        HttpRequest request = HttpRequest.newBuilder().uri(validationEndpoint)
            .header("Content-Type", "application/json")
            .POST(BodyPublishers.ofString(jsonRequest))
            .build();
        HttpClient client = HttpClient.newBuilder().build();

        return client.send(request, BodyHandlers.ofString()).body();

Where validationEndpoint is set to https://apple-pay-gateway-cert.apple.com/paymentservices/paymentSession

Again, if I instead replace the above client with the following, the request succeeds:

        char[] passwordChars = keystorePassword.toCharArray();

        KeyStore keyStore = KeyStore.getInstance("PKCS12");
        keyStore.load(new FileInputStream("/keys/MerchID.AZMVDNOW.p12"), passwordChars);

        KeyManagerFactory keyMgrFactory = KeyManagerFactory.getInstance("SunX509");
        keyMgrFactory.init(keyStore, passwordChars);

        SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
        sslContext.init(keyMgrFactory.getKeyManagers(), null, null);

        SSLParameters sslParam = new SSLParameters();
        sslParam.setNeedClientAuth(true);

        HttpClient client = HttpClient.newBuilder()
            .sslContext(sslContext)
            .sslParameters(sslParam).build();

OpenLiberty 24.0.0.6, JDK 17

So, the question is how can I get Liberty to use this specific keystore when making an https connection to the specified server?

I haven't so far found an example of making such a connection with Liberty automatically using the configured SSL configuration, so I hope I'm just missing some simple glue.

Edit:

javax.net.debug=ssl doesn't tell me anything helpful:

[err] javax.net.ssl|DEBUG|69|HttpClient-1-Worker-0|2024-07-12 20:30:31.041 UTC|SSLCipher.java:1870|KeyLimit read side: algorithm = AES/GCM/NOPADDING:KEYUPDATE
countdown value = 137438953472
[err] javax.net.ssl|DEBUG|69|HttpClient-1-Worker-0|2024-07-12 20:30:31.041 UTC|SSLCipher.java:2024|KeyLimit write side: algorithm = AES/GCM/NOPADDING:KEYUPDATE
countdown value = 137438953472
[err] javax.net.ssl|DEBUG|69|HttpClient-1-Worker-0|2024-07-12 20:30:31.051 UTC|SSLCipher.java:1870|KeyLimit read side: algorithm = AES/GCM/NOPADDING:KEYUPDATE
countdown value = 137438953472
[err] javax.net.ssl|ALL|69|HttpClient-1-Worker-0|2024-07-12 20:30:31.053 UTC|X509Authentication.java:223|No X.509 cert selected for [RSA, EC]

[err] javax.net.ssl|DEBUG|69|HttpClient-1-Worker-0|2024-07-12 20:30:31.053 UTC|SSLCipher.java:2024|KeyLimit write side: algorithm = AES/GCM/NOPADDING:KEYUPDATE
countdown value = 137438953472
[err] javax.net.ssl|DEBUG|91|Finalizer thread|2024-07-12 20:30:31.127 UTC|SSLSocketImpl.java:577|duplex close of SSLSocket

[err] javax.net.ssl|DEBUG|91|Finalizer thread|2024-07-12 20:30:31.129 UTC|SSLSocketImpl.java:1785|close the SSL connection (passive)

Edit 2:

With Liberty SSL tracing, it looks like the relevant objects loaded up successfully, including this entry showing that Liberty was able to see the certificate in the store:

[7/12/24, 20:49:04:920 UTC] 00000031 id=00000000 com.ibm.ws.ssl.config.WSKeyStore 3 alias: apple pay merchant identity certificate

And mapped the host name to the configuration:

[7/12/24, 20:49:05:241 UTC] 00000031 id=00000000 com.ibm.ws.channel.ssl.internal.SSLChannelProvider 3 setSslConfig id=applePaySSL {com.ibm.ws.ssl.internal.RepertoireConfigService, com.ibm.wsspi.ssl.SSLConfiguration}={verifyHostname=false, service.scope=bundle, component.name=com.ibm.ws.ssl.internal.RepertoireConfigService, TrustStore.target=(service.pid=com.ibm.ws.ssl.keystore_0), trustDefaultCerts=false, KeyStore.target=(service.pid=com.ibm.ws.ssl.keystore_0), sslProtocol=TLSv1.2, config.source=file, outboundConnection.0.host=apple-pay-gateway-cert.apple.com, id=applePaySSL, service.pid=com.ibm.ws.ssl.repertoire_0, clientAuthenticationSupported=false, service.id=621, keyStoreRef=com.ibm.ws.ssl.keystore_0, enforceCipherOrder=false, service.bundleid=132, outboundConnection.0.config.referenceType=com.ibm.ws.ssl.repertoire.config.outboundConnection, osgi.ds.satisfying.condition.target=(osgi.condition.id=true), config.overrides=true, clientAuthentication=true, component.id=487, effectiveTrustStore=com.ibm.ws.ssl.keystore_0, config.id=com.ibm.ws.ssl.repertoire[applePaySSL], securityLevel=HIGH, service.factoryPid=com.ibm.ws.ssl.repertoire, service.vendor=IBM, config.displayId=ssl[applePaySSL]}

But I think the outbound connection used the defaultSSLConfig instead of the applePaySSL one:

[7/12/24, 20:50:43:868 UTC] 00000046 id=00000000 com.ibm.ws.ssl.SSLPropertyUtils > lookupProperties alias=defaultSSLConfig Entry [7/12/24, 20:50:43:868 UTC] 00000046 id=00000000 com.ibm.ws.ssl.SSLPropertyUtils > getProperties sslAliasName=defaultSSLConfig currentConnectionInfo={com.ibm.ssl.direction=outbound}

I can't paste the entire huge trace here, but can provide specifics if needed.

Edit 3

On OutboundSSLSelections:

[7/15/24, 20:29:11:882 UTC] 00000031 id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 loadOutboundConnectionInfo 
                                                                                                               Adding apple-pay-gateway-cert.apple.com,* to the host list

From a later test just now, I do see these when looking up other host names:

[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 SSLConfig dynamic selection info: apple-pay-gateway-cert.apple.com,*
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 This entry has 2 attributes.
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 Host: apple-pay-gateway-cert.apple.com, Port: *
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 Host does not match.
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 No match found list
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  < lookForMatchInList Exit 
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 Cache miss tree set size is 2 entries.
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  3 No match found in host or host and port list.
[7/17/24, 12:40:30:595 UTC] 0000003b id=00000000 com.ibm.ws.ssl.config.OutboundSSLSelections                  < getPropertiesFromDynamicSelectionInfo Exit

But no entries at all for com.ibm.ssl.remoteHost with the apple host.

Solution

Based on previous answers and comments, I confirm that this works, but with a dependence on WebSphere-specific classes:

        SSLContext sslContext =
            JSSEHelper.getInstance().getSSLContext("applePaySSL", null, null);

        HttpClient client = HttpClient.newBuilder()
            .sslContext(sslContext).build();

(Where I also had to add trustDefaultCerts="true" to the <ssl id="applePaySSL" > element shown above in server.xml.)

To remove that dependence from the Java code, I looked for a way to inject the generic SSLContext instance instead. Here's what worked for our Spring DI XML configuration:

    <bean id="libertyJSSEHelper" class="com.ibm.websphere.ssl.JSSEHelper" factory-method="getInstance"/>

    <bean id="appleSSLContext" factory-bean="libertyJSSEHelper" factory-method="getSSLContext">
        <constructor-arg index="0" value="applePaySSL"/>
        <constructor-arg index="1"><null/></constructor-arg>
        <constructor-arg index="2"><null/></constructor-arg>
    </bean>

Reducing the above code to

        HttpClient client = HttpClient.newBuilder()
            .sslContext(this.getSslContext()).build();