Simultaneous client-certificate and username authentication of a SOAP web service under WCF

I have trouble authenticating to a SOAP web service, which requires simultaneous authentication with a client certificate and a username.

The target service, basically, supports three modes of authentication:

username + password — this works for me well, using BasicHttpsSecurityMode.Transport
certificate-only — can't use this mode for various reasons
certificate + username — (note that there's no password in this case, it's left blank) I can't get this mode working.

In each case, it's HTTPS with TLS 1.2.

In fact, I don't know how to configure the binding and authentication settings to describe such authentication mode in WCF. So far I experimented with various BasicHttpsBinding configurations and CustomBinding setups including that described in this answer.

In the code below, when both endpointConfig.Username and endpointConfig.ClientCertificate are specified, I build a CustomBinding based on examples I was able to google, but it won't work. The client does send the correct certificate to the server, the server replies with a Change Cipher Spec message, and the client terminates the TLS connection.

The farthest I could get is the following error messages:

Could not establish trust relationship for the SSL/TLS secure channel with authority 'ws1c.czebox.cz'.

Could not establish secure channel for SSL/TLS with authority 'ws1c.czebox.cz'.

The code for constructing the service client looks like this:

// prepare binding
Binding binding;
// NOTE A username alone is sufficient for Basic authentication e.g. when logging-in to a DataBox using the HostCert method
if (!String.IsNullOrEmpty(endpointConfig.Username) && endpointConfig.ClientCertificate != null)
{
    // client certificate and username + password
    // the below code is experimental and just doesn't work (yields the first error mentioned above)
    var tb = new HttpsTransportBindingElement();
    tb.MaxReceivedMessageSize = MaxSupportedMessageSize;
    tb.RequireClientCertificate = true;
    tb.AuthenticationScheme = AuthenticationSchemes.Basic;
    var ub = SecurityBindingElement.CreateUserNameOverTransportBindingElement();
    binding = new CustomBinding(ub, tb);
}
else if (!String.IsNullOrEmpty(endpointConfig.Username))
{
    // username + password
    var b = new BasicHttpsBinding(BasicHttpsSecurityMode.Transport);
    b.MaxReceivedMessageSize = MaxSupportedMessageSize;
    b.Security.Transport.ClientCredentialType = HttpClientCredentialType.Basic;
    binding = b;
}
else if (endpointConfig.ClientCertificate != null)
{
    // client certificate
    var b = new BasicHttpsBinding(BasicHttpsSecurityMode.Transport);
    b.MaxReceivedMessageSize = MaxSupportedMessageSize;
    b.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;
    binding = b;
}
else
{
    throw new NotSupportedException();
}

// prepare endpoint identity
EndpointIdentity identity = null;
X509Certificate2 cert = null;
if (endpointConfig.ClientCertificate != null)
{
    var clientCredentials = new ClientCredentials();
    clientCredentials.ClientCertificate.SetCertificate(
        endpointConfig.ClientCertificate.StoreLocation,
        endpointConfig.ClientCertificate.StoreName,
        endpointConfig.ClientCertificate.FindBy,
        endpointConfig.ClientCertificate.FindValue);
    cert = clientCredentials.ClientCertificate.Certificate;
    identity = EndpointIdentity.CreateX509CertificateIdentity(cert);
}

// prepare endpoint address
var address = new EndpointAddress(new Uri(endpointConfig.EndpointUrl), identity);

// create connector 
var connector = new dmInfoPortTypeClient(binding, address);

// setup username + password authentication
// NOTE Password may be empty when logging-in to a DataBox using the HostCert method
if (!String.IsNullOrEmpty(endpointConfig.Username))
{
    connector.ClientCredentials.UserName.UserName = endpointConfig.Username;
    connector.ClientCredentials.UserName.Password = endpointConfig.Password;
}

// setup client-certificate authentication
if (endpointConfig.ClientCertificate != null)
{
    connector.ClientCredentials.ClientCertificate.Certificate = cert;
}

Note that I have zero control of the server. It's a government service, can't negotiate anything.

Running on .NET 4.7.x. Can upgrade to 4.8, but can't move to .NET Core.

Solution

(Self-answering with the solution I've figured out based on extensive debugging and diagnostics. Better answers are welcome.)

It turns out the situation is almost the same as described in this Q&A pair. However, in my case I am building the WCF endpoint programmatically, avoiding XML configuration.

1) Correct custom binding

For the certificate + username option, the following code constructs a working custom binding:

// HTTPS transport with basic authentication and required client certificate
// note that AuthenticationSchemes.Certificate doesn't work, the server requires
// Basic authentication
var tb = new HttpsTransportBindingElement();
tb.MaxReceivedMessageSize = MaxSupportedMessageSize;
tb.RequireClientCertificate = true;
tb.AuthenticationScheme = AuthenticationSchemes.Basic;

// encoding: the server uses SOAP 1.1. which results in the MIME content-type text/xml
// note that by default, the encoding is SOAP 1.2, which implies the content-type
// application/soap+xml not compatible with the server, resulting in an exception
var enc = new TextMessageEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8);

// transport security: pass username over transport; it is necessary to allow
// unsecured responses; don't know why, but without this setting it won't work
var ub = SecurityBindingElement.CreateUserNameOverTransportBindingElement();
ub.EnableUnsecuredResponse = true;

// construct the binding
binding = new CustomBinding(enc, ub, tb);

Note: The are two aspects, which are, IMO, custom to my specific case, so don't fit in a general solution:

using an explicit encoding denoting MessageVersion.Soap11 for SOAP 1.1; by default, the encoding is MessageVersion.Soap12 for SOAP 1.2
enabling unsecured responses using the SecurityBindingElement.EnableUnsecuredResponse

2) No endpoint identity

The code section commented with // prepare endpoint identity was wrong, incorrectly using the client-side certificate as the one WCF is supposed to expect to receive from the server.

3) WCF tracing

When diagnosing the problem, what enabling WCF tracing was instrumental. The following can be used in app.config to enable logging:

<system.diagnostics>
    <!-- use for WCF diagnostis: WCF tracing and message logging -->
    <sources>
        <source name="System.ServiceModel"
        switchValue="Information, ActivityTracing"
        propagateActivity="true">
        <listeners>
            <add name="traceListener"
            type="System.Diagnostics.XmlWriterTraceListener"
            initializeData= "D:\Logs\WCF-Traces.svclog" />
        </listeners>
        </source>
        <source name="System.ServiceModel.MessageLogging">
        <listeners>
            <add name="messages"
            type="System.Diagnostics.XmlWriterTraceListener"
            initializeData="D:\Logs\WCF-Messages.svclog" />
        </listeners>
        </source>
    </sources>
</system.diagnostics>