Search code examples
securityssl-certificateibm-mqtls1.2

MQ Client -> MQ Server TLS Communication fails with different errorrs AMQ9665E


THis is my first question here so please excuse the possible format mistakes or so.

I'm trying to setup a TLS connection between MQ Server and MQ Clients as follows:

MQ Server 
I create a new key.kdb  and imported a .pfx certificate signed by our internal CA - ROOT CA -Intermediate CA-Issuing CA (these signers automatically get imported with the .pfx) 
-personal certificate label renamed to ibmwebspheremqqname

MQ Clients:
We have cluster of 6 Biztalk servers 

So imported .pfx certificate same way, issued by exact same authorities.
-personal certificate label renamed to ibmwebspheremqserviceaccount (service account used for connection) 

Server side everything is configured on all params because in this test environmnet we used to have TLS but with Self Signed Certificates and it worked. Actually, when I am importing the SelfSigned cert of one of the Client MQ servers on MQ server side, connection is established. I have no idea how because I removed everything from Client side from key.kdb.

Errors I get on Clients side:

1/11/2024 12:58:28 - Process(14980.2) User(serviceAccount) Program(BTSNTSvc.exe)
                      Host(Host) Installation(Installation1)
                      VRMF(9.2.0.7)
                      Time(2024-01-11T11:58:28.800Z)
                      RemoteHost(IP)
                      ArithInsert1(420)
                      CommentInsert1(Channel)
                      CommentInsert2(gsk_secure_soc_init)
                      CommentInsert3(MQSERVER (IP)(Port))
                     
AMQ9665E: SSL connection closed by remote end of channel
'NIBC.CH.STRGTLS.CLAU'.

EXPLANATION:
The SSL or TLS connection was closed by the remote host 'MQSERVER
(IP)(Port)' during the secure socket handshake. The channel is
'Channel'; in some cases its name cannot be determined and so is
shown as '????'. The channel did not start.
ACTION:
Check the remote end of the channel for SSL and TLS errors. Fix them and
restart the channel. 

+
AMQ9208E: Error on receive from host MQSERVER(IP)(Port).

EXPLANATION:
An error occurred receiving data from MQSERVER(IP)(PORT) over
TCP/IP. This may be due to a communications failure.

Errors I get on MQ Server side:

AMQ9999E: Channel '????' to host 'client (ip)' ended abnormally.

EXPLANATION:
The channel program running under process ID 81092(11660) for channel '????'
ended abnormally. The host name is 'client (IP)'; in some cases
the host name cannot be determined and so is shown as '????'.

and: 

The channel is '????'; in some cases its name cannot be determined and so is
shown as '????'. The remote host is 'client (ip)'. The channel
did not start. 

The details of the certificate which could not be validated are
'[Class=]GSKVALMethod::X509[Issuer=]CN=btsacpm432.prd.domain,C=US[#=]613b2cab[Subject=]CN=client.prd.domain,C=US[Class=]GSKVALMethod::PKIX[Issuer=]CN=client.prd.domain,C=US[#=]613b2cab[Subject=]CN=client.prd.domain,C=US'.


The certificate validation error was 575010.
ACTION:
Check which of the possible causes applies on your system. Correct the error,
and restart the channel. ` 

I have already changed the QM SSL property to point to the new key.kdb instead of the old one. It looks like for somea reason is checking the old self signed certificate.

I also want to mention that in Production we already have this solution in place and its working exactly like I explained in the first part with the two certificates signed by internal CA.

Weirdly enought managed to make one of the 6 cluster servers to actually work with this configuration. All the others are not.

I am testing the TLS connection with this

C:\Program Files\IBM\MQ\bin>AMQSSSLC -m QM -c Channel -x ip(port) -k "C:\Program Files\IBM\MQ\key" -s TLS_RSA_WITH_AES_128_CBC_SHA256 -l ibmwebspheremqserviceaccount
Sample AMQSSSLC start
Connecting to queue manager QM
Using the server connection channel TLS CHANNEL
on connection name IP(1418).
Using SSL CipherSpec TLS_RSA_WITH_AES_128_CBC_SHA256
Using SSL key repository stem C:\Program Files\IBM\MQ\key
Certificate Label: ibmwebspheremqserviceaccount
No OCSP configuration specified.
Connection established to queue manager QM
Sample AMQSSSLC end

But the same its working on the other 5 servers. So I guess this is irrelevant at this point?.


Solution

  • I am not 100% sure what the problem was, but there are two possible reasons:

    1) I had two QMs on MQ Server that communicates with same Biztalk MQ Clients.

    One QM was configured with Self Signed Certs, both ways handshake so MQ key.kdb contained Biztalk's Self Signed Certs in signers, and all Biztalk MQ Clients contained the Self Signed of MQ Server for QM.

    Second QM, handshake based on CA certs like explained in the first part of the post.

    2) Biztalks MQC side, in my case it has no specific configuration in .ini file, (like SSLKeyRepository where too look for) so maybe because it was looking directly in the installation folder, I had there the Self Signed certs among other certs.

    I eliminated both possible reasons and now communication is working as this:

    MQ Server I create a new key.kdb and imported a .pfx certificate signed by our internal CA - ROOT CA -Intermediate CA-Issuing CA (these signers automatically get imported with the .pfx) -personal certificate label renamed to ibmwebspheremqqname

    MQ Clients: We have cluster of 6 Biztalk servers So imported .pfx certificate same way, issued by exact same authorities. -personal certificate label renamed to ibmwebspheremqserviceaccount (service account used for connection)

    As it should work because I already have this kind of communication between same MQ Server and other MQ Clients.

    Thanks for suggestions and help. Best regards,

    Narcis V.