Secure handshake failing during IBM.MQ MQQueueManager on Linux .net core

Question

We are trying to connect to IBM MQ from RHEL 8.3 using .net core 3.1 application. The same code runs ok on Windows- we are able to connect to QM.
We are facing an error that comes, as we believe, from openssl:

---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL. ---> Interop+Crypto+OpenSslCryptographicException: error:14094412:SSL routines:ssl3_read_bytes:sslv3 >alert bad certificate --- End of inner exception stack trace --- at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 >recvCount, Byte[]& sendBuf, Int32& sendCount) in >/_/src/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSsl.cs:line 278

We can see that MQClient loaded certificates

00000157 12:30:07.116059 536167.15 -----------{ MQEncryptedSocket.MakeSecuredConnection()
00000158 12:30:07.116425 536167.15 Created an instance of SSLStreams
00000159 12:30:07.116466 536167.15 Setting current certificate store as 'User'
0000015A 12:30:07.116528 536167.15 Linux so use My & CurrentUser
0000015B 12:30:07.116542 536167.15 Created store object to access certificates 0000015C 12:30:07.116780 536167.15 Opened store
0000015D 12:30:07.116795 536167.15 Accessing certificate - ZZZZ
0000015E 12:30:07.230836 536167.15 Number of certificates in the store:5
0000015F 12:30:07.231092 536167.15 TLS12 supported - True
00000160 12:30:07.233622 536167.15 Setting SslProtol as Tls12
00000161 12:30:07.233652 536167.15 Starting SSL Authentication

We have unit test that verifies if cert of given subject exists in that store and it passes.
What we've done already:

1. We build openssl 1.1.1h and linked it system wide.
2. We verified cert store. The cert we are using is valid

    var cert = x509Store2.Certificates.Find(X509FindType.FindBySubjectName, mockIbmMqOptions.MqCertificateLabel, true).Count;
    x509Store2.Close();
    x509Store2.Dispose();
    Assert.True(cert > 0);

3. We ran the code on Windows with success.
4. We searched for sslv3 alert bad certificate in openssl source and found nothing but statics with error code 1420 but not referenced.

Any ideas?

Answer 1

Thank you @JoshMc based on this question we successfully managed to finish the handshake. I think this is a general solution for .net core on RHEL and IBM MQ Client libraries:

1. Be sure that CA and Intermediate CA are stored in /usr/share/pki/ca-trust-source/anchors/ or /etc/pki/ca-trust/source/anchors/
2. If your pfx contain certificates in certification path EXPORT ONLY private key and public cert
3. Be sure to have Friendly Name set to some value.
4. Add exported pfx to .net core key store certificate-tool add

5. Set CertificateLabel to match FriendlyName or set the friendly name to ibmwebspheremq<username_in_lowercase>