Search code examples
curlurlminiopre-signed-url

Why would be an IP address in URL enclosed in square brackets and what HTTP host header is sent?


I have a really specific issue that I thought is a bug, but it appears not. I am using minIO client to generate presigned URL for AWS S3 object. What I get from that tool is an URL that starts like this:

http://[192.168.0.1]/bucket_name/object_name

Thinking it is a bug, I replaced the brackets with regular expression, but then the URL would not work (hostname is part of the AWS signature). Looking deeper into it, I see this in URL specification (RFC 3986):

The host subcomponent of authority is identified by an IP literal encapsulated within square brackets, an IPv4 address in dotted- decimal form, or a registered name.

Never heard of this. If I pass this to curl -g I get an error, as I'd expect:

curl: (3) URL using bad/illegal format or missing URL

Curiously, after a lot of digging I know for a fact that at some point in time, it worked.

So the question is basically, is this really valid and if it is, what does the corresponding Host header look like for such URL. I need to fetch from that URL without altering it.


Solution

  • If you read a bit further down in RFC 3986§3.2.2, the only valid IP-literal is one that contains an IPv6 (or later) address:

    IP-literal = "[" ( IPv6address / IPvFuture  ) "]"
    

    A bit before that, there is explicit verbiage indicating these brackets are only applicable for instances where an IPv6 address is present as the Host:

    A host identified by an Internet Protocol literal address, version 6 [RFC3513] or later, is distinguished by enclosing the IP literal within square brackets ("[" and "]"). This is the only place where square bracket characters are allowed in the URI syntax.

    Unless the tool usually returns an IPv6 address, this is likely indeed a bug (or some sort of misconfiguration somewhere).