HttpWebRequest throws "The remote server returned an error: (404) Not Found." for 404

Question

I am trying to issue a web-request:

String url = @"http://stackoverflow.com/aaaaaaaa.html";

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

String responseText;

using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
   if (response.StatusCode == HttpStatusCode.NotFound) //404
      return "";

   using (StreamReader responseStream = new StreamReader(response.GetResponseStream()))
   {
      responseText = responseStream.ReadToEnd();
   }
}

The problem is that the line:

request.GetResponse();

throws an exception:

• System.Net.HttpWebResponse
  • "The remote server returned an error: (404) Not Found."

Why is HttpWebRequest throwing an exception; and how do i get it to stop?

Alternatively

Can someone suggest a method, or .NET class, that can let me talk to a web-server, and retrieve the perfectly valid not exceptional 404.

HttpClient has its own issues that remove it from consideration.

Answer 1

Here's some code that I wrote before that you're welcome to use that deals with this scenario. If an exception is thrown, you can get the status code by catching the WebException.

public class WebResult
{
    public string Response { get; set; }
    public bool WasSuccessful { get; set; }
    public HttpStatusCode? StatusCode { get; set; }
}

public WebResult GetUrlContents(string url)
{
    try
    {
        var request = (HttpWebRequest)WebRequest.Create(url);

        using (var response = (HttpWebResponse)request.GetResponse())
        using (var responseStream = new StreamReader(response.GetResponseStream()))
        {
            return new WebResult
            {
                WasSuccessful = true,
                Response = responseStream.ReadToEnd(),
                StatusCode = response.StatusCode
            };
        }
    }
    catch (WebException webException)
    {
        return new WebResult
        {
            Response = null,
            WasSuccessful = false,
            StatusCode = (webException.Response as HttpWebResponse)?.StatusCode
        };
    }
}