It has been years since this issue was first raised on the Internet, and I still do not see that anyone has solved it.
WSE had the ability to set a flag Security.EncodedMustUnderstand = "0". WCF no longer seems to have that capability. A custom message inspector never sees the security header in the BeforeSendRequest method, and any changes made to headers in it do not seem to take affect anyway.
This appears to be a massive interoperability issue with either legacy services, or java services. Why has it not yet been resolved by Microsoft?
Does anyone have a workaround yet for this issue?
In all of my searches, only one person has actually claimed to solve the issue here. But their fix must not apply in my scenario because it does not modify the outgoing message at all.
I am using a CustomBinding and require x509 cert and username security
var security = SecurityBindingElement.CreateCertificateOverTransportBindingElement();
//var security = new TransportSecurityBindingElement();
security.IncludeTimestamp = true;
security.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic256;
security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10;
//security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11;
//security.EndpointSupportingTokenParameters.SignedEncrypted.Add(new X509SecurityTokenParameters(X509KeyIdentifierClauseType.Any, SecurityTokenInclusionMode.AlwaysToRecipient)); // add specific x509 cert security security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
security.SecurityHeaderLayout = SecurityHeaderLayout.Lax;
security.EnableUnsecuredResponse = true;
//security.ProtectTokens = false;
//security.DefaultAlgorithmSuite = new Basic128Sha256Rsa15Sha1AlgorithmSuite(); // when we need to tweak the security suite
var encoding = new TextMessageEncodingBindingElement();
//encoding.MessageVersion = MessageVersion.CreateVersion(EnvelopeVersion.Soap11, AddressingVersion.None);// MessageVersion.Soap11;
encoding.MessageVersion = MessageVersion.Soap11WSAddressingAugust2004;
encoding.WriteEncoding = Encoding.UTF8;
var transport = new HttpsTransportBindingElement();
transport.MaxReceivedMessageSize = 20000000; // 20 megs
transport.RequireClientCertificate = false;
CustomBinding binding = new CustomBinding();
binding.Elements.Add(security);
binding.Elements.Add(encoding);
//binding.Elements.Add(new TimestampedTextMsgEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8));
binding.Elements.Add(transport);
EDIT:
I was finally able to get this half working with a custom encoder. It seems like a massive failing on Microsoft's part to remove this interoperability feature, but it can be hacked in with a custom encoder.
Unfortunately, while the service is returning what appears to be valid xml (observed in Fiddler), the CustomEncoder is blowing up with a stack overflow error before it ever hits the ReadMessage function. Still trying to figure out why.
Sometimes instead of the stack overflow error I get this chain of inner exceptions:
Message: An error occurred while making the HTTP request to https://service. This could be due to the fact that the server certificate is not configured properly with HTTP.SYS in the HTTPS case. This could also be caused by a mismatch of the security binding between the client and the server.
Message: The underlying connection was closed: An unexpected error occurred on a send.
Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
The odd part is that the connection is not being forced closed by the remote host, the remote host seems to think everything was successful. So this is an error inside of .net. Either a problem with .net itself, or an issue with my custom encoder which was copied directly from microsoft's site.
I have finally been able to figure things out. At the root of this problem is Microsoft removing the Security.EncodedMustUnderstand = "0" feature from WCF. Whatever reason they chose to do that for it caused a lot of extra work on my end. Creating a custom encoder that manually modified this setting was the only way I could find to work around that issue.
My second big issue came when I copied Microsoft's example code off of their site for the custom encoder. Turns out there is a hidden loop in it that generates a stack overflow error.
Once I cleaned up Microsoft's example code I was still being presented with the same HTTP.SYS errors. This time it was actually a valid error, and moving the code to a server where the call could be sourced from an https url solved the issue.
Edit: I wrote a detailed blog post about this answer here.