HttpClient request with x509 Certificate works on Debug but fails in Production (403 forbidden)

I have a black box service I have to call into with simple rest commands that returns xml.

They issued us a certificate that had to be run in IE and installs in to IE's Certificate section. As per their instructions I exported it with the entire chain as a pfx with password.

On the machine that the cert was issued directly to, everything works fine in code

        var certHandler = new WebRequestHandler();
        certHandler.ClientCertificateOptions = ClientCertificateOption.Manual;            
        certHandler.UseDefaultCredentials = false;

        var certificate = new X509Certificate2(Properties.Resources.SigningCert, "password", X509KeyStorageFlags.DefaultKeySet | X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet); //Must be renewed and replaced every year.
        certHandler.ClientCertificates.Add(certificate);            

        //Execute the command            
        var client = new HttpClient(certHandler);
        string result;
        try
        {
            result = await client.GetStringAsync(url);

            System.Diagnostics.Debug.WriteLine(result);
        }
        catch (Exception ex)
        {
            throw ex;
        }

(I've stored the cert in the resources, but it loads fine and loading it from a file works fine too in developer machine.) I also imported it into IE on the server just in case. Obviously this is likely under the wrong cert store, but I couldn't figure out how to load this in globally. I can tell you that the same REST GETs work in IE on the server just like they do on the developer machine. It's only in code that it fails.)

In production, this same code throws a 403 forbidden.

Production (really a beta server) is actually behind the same nat as the as the development machine so they're seeing the same IP come through etc.

Any ideas why it would fail on the server and not on the developer box?

Thanks!

Solution

You should use X509KeyStorageFlags according to account under which your app is running. If it is

1) An app that runs under regular Windows User Account you should use

X509Certificate2(Properties.Resources.SigningCert, "password", X509KeyStorageFlags.UserKeySet);

2) Windows Service under LocalSystem, IIS under NetworkService or other services under built in Windows Account, you should use

X509Certificate2(Properties.Resources.SigningCert, "password", X509KeyStorageFlags.MachineKeySet);

Basically, you shouldn't use X509KeyStorageFlags.PersistKeySet in your case - you import certificate from pfx format every time.

Certificate's private key is storing in the container according to flags. So you may have no access to it if you use wrong flags. DefaultKeySet is not just alias for UserKeySet (msdn) - so choose appropriate flags in every case.

These articles also may be helpfull: