I have a web app using HttpWebRequest to send some data to a third party service. It is failing when I run it through the web code but when I send pretty much the exact same data using Fiddler's composer it works.
I say "pretty much" because the data contains a password digest and nonce and timestamp that change for every message. I have put a break point and taken the data the web app was about to send and pasted it into Fiddler and it works.
I have set up my code to run through Fiddler's proxy too, which is great - I can see the "raw" data being sent - it still fails, but it's enabled me to check that the headers are all exactly the same and mess with them a bit (didn't find anything useful).
WebProxy myproxy = new WebProxy("127.0.0.1:8889", false);
req.Proxy = myproxy;
I've disabled Fiddler's "automatically authenticate" and "follow redirects" options (have left on "fix content-length header"), just in case one of those was causing Fiddler to act smart.
This post is to a https URL so I did have to add the following lines to ignore Fiddler's certificate problems when decrypting the HTTPS traffic
ServicePointManager
.ServerCertificateValidationCallback +=
(sender, cert, chain, sslPolicyErrors) => true;
Does anyone know of any options on HttpWebRequest that I should enable or disable to make it act as Fiddler does? As the actual content seems to be same, maybe there's a difference in the security/authentication options?
Changing Fiddler's "decrypt HTTPS" setting doesn't seem to affect anything, except mean I can't view the data.
Would using something other than HttpWebRequest be an option? What other libraries are there that don't just use HttpWebRequest under the hood? I don't need async but does HttpClient use a newer library that handles things better?
I've seen other posts about connections and making HttpWebRequest more performant, but at the moment I'm just POSTing a single message.
If it helps here's what the headers look like:
POST https://www.server.com/enterprise/soap?ServiceName=PassportService&auth=1 HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: http://www.server.com/ws/passport/2008/04/PassportService#publishDocumentWithParameters
Host: vha.server.com
Content-Length: 4601
Expect: 100-continue
Connection: Keep-Alive
the body of the message is just SOAP xml
Here is some data from implementing .NET Network Tracing
System.Net Information: 0 : [8864] ConnectStream#7307007 - Sending headers
{
Content-Type: text/xml; charset=utf-8
SOAPAction: http://www.server.com/ws/passport/2008/04/PassportService#publishDocumentWithParameters
Host: vha.server.com
Content-Length: 4393
Expect: 100-continue
Connection: Keep-Alive
}.
System.Net Information: 0 : [8864] SecureChannel#63848051::.ctor(hostname=vha.server.com, #clientCertificates=0, encryptionPolicy=RequireEncryption)
System.Net Information: 0 : [8864] Enumerating security packages:
System.Net Information: 0 : [8864] Negotiate
System.Net Information: 0 : [8864] NegoExtender
System.Net Information: 0 : [8864] Kerberos
System.Net Information: 0 : [8864] NTLM
System.Net Information: 0 : [8864] Schannel
System.Net Information: 0 : [8864] Microsoft Unified Security Protocol Provider
System.Net Information: 0 : [8864] WDigest
System.Net Information: 0 : [8864] TSSSP
System.Net Information: 0 : [8864] pku2u
System.Net Information: 0 : [8864] CREDSSP
System.Net Information: 0 : [8864] SecureChannel#63848051 - Left with 0 client certificates to choose from.
System.Net Information: 0 : [8864] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent = Outbound, scc = System.Net.SecureCredential)
System.Net Information: 0 : [8864] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = vha.server.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [8864] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=112, returned code=ContinueNeeded).
System.Net.Sockets Verbose: 0 : [8864] Socket#55285825::Send()
... then there is some certificate back and forth ...
System.Net Information: 0 : [8864] SecureChannel#63848051 - Remote certificate was verified as valid by the user.
System.Net Information: 0 : [8864] ProcessAuthentication(Protocol=Tls, Cipher=Rc4 128 bit strength, Hash=Md5 128 bit strength, Key Exchange=RsaKeyX 2048 bit strength).
System.Net.Sockets Verbose: 0 : [8864] Socket#55285825::Send()
... then some more stuff ...
System.Net Verbose: 0 : [8864] Exiting HttpWebRequest#49916336::GetRequestStream() -> ConnectStream#7307007
System.Net Verbose: 0 : [8864] ConnectStream#7307007::Write()
System.Net Verbose: 0 : [8864] Data from ConnectStream#7307007::Write
System.Net Verbose: 0 : [8864] 00000000 : 3C 73 6F 61 70 65 6E 76-3A 45 6E 76 65 6C 6F 70 : <soapenv:Envelop
... more SOAP data follows (and what looks like encrypted version ... ... encrypted reply comes back, then it's decrypted ...
System.Net Information: 0 : [8864] Connection#4562529 - Received status line: Version=1.1, StatusCode=401, StatusDescription=Unauthorized.
System.Net Information: 0 : [8864] Connection#4562529 - Received headers
{
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Length: 666
Content-Type: text/xml;charset=UTF-8
Date: Thu, 07 Aug 2014 06:21:40 GMT
Server: XXXX Web Server 8
}.
System.Net Information: 0 : [8864] ConnectStream#27021036::ConnectStream(Buffered 666 bytes.)
System.Net Information: 0 : [8864] Associating HttpWebRequest#49916336 with ConnectStream#27021036
System.Net Information: 0 : [8864] Associating HttpWebRequest#49916336 with HttpWebResponse#16709290
... with some XML with some error info (that the 3rd party doesn't seem to be able to use to pinpoint the issue ...
The short answer is that there is no difference.
Thanks to help from @EricLaw I was able to rule out any other variables he could think of.
The problem was the timestamp in the password digest. My machine time was slightly off and theirs was too (but the other way). So the result was that it worked when I pasted the data into Fiddler because I was delaying it a bit (but not too much). There's a 37 second window for these requests, and when it ran from the code it looked like it was a time from the future to their server.
When there's only one variable, that's probably the problem!
Thanks for your patience and help Eric.