VBA ServerXMLHTTP proxy is fine with https but fails on http

Question

I'm trying to write a macro to do some querying from the web to update an Access database. For some reason, VBA refuses to play friendly with http, but is totally content to do https.

Here's my requesting function:

Function httpRequest(ByVal url As String, useProxy As Boolean) As String

    Dim response As String
    Dim proxy As String
    Dim xhr As Object


    'Make HTTP requester object
    Set xhr = CreateObject("MSXML2.ServerXMLHTTP.6.0")

    If useProxy Then
        If Left(url, 5) = "https" Then
            proxy = "proxyb1secure:8443"
        Else
            proxy = "proxyb1:8080"
        End If
        xhr.setProxy 2, proxy
    End If

    xhr.Open "GET", url, False

    'send the request. THIS LINE TIMES OUT ON HTTP
    xhr.Send
    'fetch the whole page
    response = xhr.responseText

    'clean up
    Set xhr = Nothing

    'return
    httpRequest = response

End Function

And my testing function:

Function testProxy()
    'This one works
    MsgBox (httpRequest("https://www.bing.com/search?q=stackoverflow", True))
    'This one doesn't.
    MsgBox (httpRequest("http://www.bing.com/search?q=stackoverflow", True))
End Function

I'm certain I'm going after the right name and port, because I've tested the same thing via Java, and it's content to do both flavors (i.e. everything works perfectly in the following code).

public static void main(String[] args) throws Exception {
    URL url = new URL("http://www.bing.com/search?q=stackoverflow");
    HttpURLConnection con = (HttpURLConnection) url.openConnection(getProxyForURL(url));
    System.out.println(con.getResponseCode() + " " + con.getResponseMessage());
    InputStream is = con.getInputStream();
    int c;
    StringBuilder sb = new StringBuilder();
    while ((c = is.read()) != -1) {
        sb.append((char) c);
    }
    String page = sb.toString();
    System.out.println(page);
}

public static Proxy getProxyForURL(URL url) {
    if (url.getProtocol().equals("https")) {
        return new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxyb1secure", 8443));
    } else {
        return new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxyb1", 8080));
    }
}

What trickery of VBA am I missing?

Answer 1

Mystery solved. Turns out this is a security feature revolving around the user agent (of all things...).

Java used these HTTP headers (which were successful):

GET http://www.bing.com/search?q=stackoverflow HTTP/1.1
Accept: */*
Accept-Language: en-us
Proxy-Connection: Keep-Alive
User-Agent: Java/1.7.0_79
Host: www.bing.com

and Access sent these (which were unsuccessful):

GET http://www.bing.com/search?q=stackoverflow HTTP/1.1
Accept: */*
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)
Proxy-Connection: Keep-Alive
Host: www.bing.com

by simply adding a

xhr.setRequestHeader "User-Agent", "PayMeNoAttention"

it magically goes through. And to confirm the theory, adding Access' user-agent to Java caused it to fail. So. That's definitely what's up.

This is likely an attempt by our brilliant network techs to block macro viruses from contacting malicious sites.