Search code examples
c#.nethosts

System.Net.WebRequest not respecting hosts file

Is there a way to get a System.Net.WebRequest or System.Net.WebClient to respect the hosts or lmhosts file?

For example: in my hosts file I have:

10.0.0.1  www.bing.com

When I try to load Bing in a browser (both IE and FF) it fails to load as expected.

Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1 
WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected)
WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds

Similarly:

WebClient wc = new WebClient();
wc.DownloadString("http://www.bing.com"); //succeeds

Why would System.Net.Dns respect the hosts file but System.Net.WebRequest ignore it? What do I need to change to make the WebRequest respect the hosts file?

Additional Info:

  • If I disable IPv6 and set my IPv4 DNS Server to 127.0.0.1, the above code works (fails) as expected. However if I add my normal DNS servers back as alternates, the unexpected behavior resumes.
  • I've reproduced this on 3 Win7 and 2 Vista boxes. The only constant is my company's network.
  • I'm using .NET 3.5 SP1 and VS2008

Edit

Per @Richard Beier's suggestion, I tried out System.Net tracing. With tracing ON the WebRequest fails as it should. However as soon as I turn tracing OFF the behavior reverts to the unexpected success. I have reproduced this on the same machines as before in both debug and release mode.

Edit 2

This turned out to be the company proxy giving us issues. Our solution was a custom proxy config script for our test machines that had "bing.com" point to DIRECT instead of the default proxy.

Solution

  • I think that @Hans Passant has spotted the issue here. It looks like you have a proxy setup in IE.

    Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1

    This works because you are asking the OS to get the IP addresses for www.bing.com

    WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds

    This works because you are asking the framework to fetch a path from a server name. The framework uses the same engine and settings that IE frontend uses and hence if your company has specified by a GPO that you use a company proxy server, it is that proxy server that resolves the IP address for www.bing.com rather than you.

    WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected)

    This works/fails because you have asked the framework to fetch you a webpage from a specific server (by IP). Even if you do have a proxy set, this proxy will still not be able to connect to this IP address.

    I hope that this helps.

    Jonathan