Search code examples
c#.netsslopensslsslstream

"Call to SSPI failed" when connecting to OpenSSL Server


When calling AuthenticateAsClient(), I receive the error "Call to SSPI failed." with the inner exception "The message received was unexpected or badly formatted".

I've seen this to be a semi-popular problem, but I haven't been able to find a working solution. Here is where I'm at after scouring articles for a few hours:

  • Using my private key, the server issued me a signed client cert
  • Using open ssl, I combined the two into a pfx using:

    openssl pkcs12 -in My-Client-Cert.pem -inkey ssl-client-privatekey.pem -export -out private-key-pair.pfx

  • I imported the cert to the personal folder on LocalMachine. I was able to see that cert recognized that a private key was present.
  • The private key is not password protected
  • I imported the CA Root for the service I'm talking to into both my Trusted Root Certification Authorities and Personal stores.
  • In app code, I pull the cert by thumbnail. I can see in the debugger that both private and public keys are present.
  • I initialize a TcpClient
  • I then initialize an SslStream using the the TcpClient
  • I call AuthenticateAsClient()
  • I receive the error message

For the hostname parameter of AuthenticateAsClient() I have also tried using the CN of the CA root and also the CN of the client cert with the same result.

I verified that the service I'm trying to talk to does work just fine when I connect to it using openssl s_client connect

Here is the code for the connection:

private void Setup(string hostname, int port, X509Certificate2Collection certs)
{
    try
    {
        // create the socket
        var clientSocket = new TcpClient(hostname, port);
        _sslStream = new SslStream(
            clientSocket.GetStream(),
            false,
            ValidateServerCertificate,
            null);

        _sslStream.AuthenticateAsClient(
            "VirtuCrypt Root CA - Test", // hostname here must match the name on the server certificate
            certs,
            SslProtocols.Tls11,
            false);

        Debug.WriteLine("Connected!");
    }
    catch(Exception ex)
    {
        Debug.WriteLine(ex.Message);

        if(ex.InnerException != null)
            Debug.WriteLine(ex.InnerException.Message);
    }
}

Pulling my hair out... any thoughts?

UPDATE: I spun up Wireshark and noticed that all communication to and from the host was done over TCP, and none of it was done over TLS. Not sure if that's something I should be concerned with despite the fact that I am requesting TLS12

UPDATE2: I'm looking at Wireshark again and I fixed the decode so that it shows the TLS communications in addition to the TCP connection. Now that I can see the handshake, I can see that no client cert is actually being presented. I'm providing that cert to SslStream, so I don't know why it's not being transmitted.


Solution

  • Ok after finally making friends with Wireshark and breaking about 4 coffee mugs chasing my tail, here's what I found out...

    • The CA root that was given to us was a pem file that actually included the CA root and 3 intermediate certs.
    • I had to break those into separate files and import the CA root into our trusted root CA store and the 3 intermediate certs into the intermediate CA store
    • I hadn't noticed before, but our client cert that was installed in the personal store was saying it didn't have enough info to be validated, but what was really going on is that it was failing validation because we didn't have the CA certs installed correctly as described above. Once I installed those separately, this message went away
    • It appears that .Net will not send your client cert unless it can be validated. In my 2nd edit in the OP, I mentioned that I could see the client cert was not being sent at all. This was the actual problem. Once I installed these CA certs, the cert started getting sent and everything else was good.