Search code examples
javacurlhttpshttp-headershttpsurlconnection

Why Java's HttpsURLConnection and curl's HEAD requests return different headers?


I noticed that javax.net.ssl.HttpsURLConnection miss some headers in a response when I am using HEAD request.

Here is the code I use:

  public static void getHeaders() throws IOException, URISyntaxException {
        URL url = new URL("https://acme-v01.api.letsencrypt.org/terms");
        HttpsURLConnection con = (HttpsURLConnection)url.openConnection();
        con.setRequestMethod("HEAD");
        con.getInputStream().close();
        System.out.println(con.getHeaderField("Location"));
        Map<String, List<String>> map = con.getHeaderFields();
        for (Map.Entry<String, List<String>> entry : map.entrySet()) {
            System.out.println("Key : " + entry.getKey() +
                    " ,Value : " + entry.getValue());
        }
    }

Output is the next:

null
Key : X-Frame-Options ,Value : [DENY]
Key : null ,Value : [HTTP/1.1 200 OK]
Key : Cache-Control ,Value : [max-age=60987]
Key : Server ,Value : [nginx]
Key : ETag ,Value : ["582cdb9c-22004"]
Key : Connection ,Value : [keep-alive]
Key : Expires ,Value : [Thu, 01 Dec 2016 14:26:25 GMT]
Key : Last-Modified ,Value : [Wed, 16 Nov 2016 22:20:12 GMT]
Key : Date ,Value : [Wed, 30 Nov 2016 21:29:58 GMT]
Key : Content-Type ,Value : [application/pdf]

We can see that it didn't return "Location" header.

Here is a simple curl request executed on the same server:

# curl -I https://acme-v01.api.letsencrypt.org/terms
HTTP/1.1 302 Moved Temporarily
Server: nginx
Content-Type: text/plain; charset=utf-8
Boulder-Request-Id: JBtoTNb6Q-vvdR5VF9j3gIP27S4RmEqaLjepyF60aVU
Location: https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf
Replay-Nonce: x7jscxXZ-90jzgLG8vkmunp7XrlLrMT0R4mzFV7xAhQ
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Content-Length: 0
Expires: Wed, 30 Nov 2016 21:37:03 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Wed, 30 Nov 2016 21:37:03 GMT
Connection: keep-alive

Why equal "HEAD" requests return different headers?


Solution

  • As zapl said in comments it was because this link has a redirection. And seems that by default HttpsURLConnection accepts redirects. But we can manually turn it off:

    con.setInstanceFollowRedirects(false);