Search code examples
c#httphttpwebrequestfiddler

Server returns 421 HTTP error when Fiddler is not running

I'm building an API for Pixiv and so far I've managed to complete the login process and searching by tags works as well.

Now I want to download the image on the selected webpage. At first I was getting some certificate errors. I fixed that by temporarily ignoring certificate validation.

Then a very interesting thing happened... If Fiddler was running in the background - the response was successfully received and I was able to download the image. No problems, whatsoever.

But when the Fiddler was NOT running, the remote server returned the 421 error code.

Things I've already tried:

  • Closing every response manually AND using the "using" statement
  • Enforced the Keep-Alive header.
  • Explicitly declared the use of HTTP/1.1
  • Changing proxy settings on the HttpWebRequest object

My image request code looks like this:

HttpWebRequest imagerequest = (HttpWebRequest)WebRequest.Create(imgLink);
imagerequest.Host = $"i.pximg.net";
imagerequest.UserAgent = USER_AGENT;
imagerequest.Accept = "image/webp,image/*,*/*;q=0.8";
imagerequest.Referer = post.PostLink;
imagerequest.Headers.Add("Accept-Encoding", "gzip, deflate, sdch, br");
imagerequest.Headers.Add("Accept-Language", "en-US,en;q=0.8,sl;q=0.6");

byte[] imagecontent;
using (var imageresponse = (HttpWebResponse)imagerequest.GetResponse())
{
    using (var imagestream = imageresponse.GetResponseStream())
    {
        imagecontent = imagestream.ReadStream();
    }
}

(I also send a request before this one in order to get the image link)

Example of "imgLink":

https://i3.pixiv.net/img-original/img/2014/02/09/00/10/03/41485846_p0.jpg

Fiddler displays the sent request as this:

GET https://i.pximg.net/img-original/img/2014/02/09/00/10/03/41485846_p0.jpg   HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML,   like Gecko) Chrome/54.0.2840.99 Safari/537.36
Accept: image/webp,image/*,*/*;q=0.8
Referer: http://www.pixiv.net/whitecube/user/2824699/illust/41485846
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8,sl;q=0.6
Host: i.pximg.net

Fiddler does not show any signs of HTTP protocol violations.

Solution

  • Seems like I was just blind. The "imgLink" had the wrong hostname set the entire time, but I didn't notice it because the hostnames were so similar. I guess Fiddler was automatically fixing that for me...

    Next time I will be sure to triple-check the hostname. Thanks for for all the help.