Use urllib in python to fetch IPv6 URL with FQDN

Question

I want to fetch an IPv6 page with urllib. Works with square brack IPv6 notation but I have no clue how to (easily) convince python to do an IPv6 request when I give it the FQDN Like the below ip is: https://www.dslreports.com/whatismyip

from sys import version_info

PY3K = version_info >= (3, 0)

if PY3K:
    import urllib.request as urllib
else:
    import urllib2 as urllib

url = None
opener = urllib.build_opener()
opener.addheaders = [('User-agent',
     "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36")]
url = opener.open("http://[2607:fad0:3706:1::1000]/whatismyip", timeout=3)
content = url.read()

Answer 1

I finally solved my issue. Not in the most elegant way, but it works for me.

After reading:

Force requests to use IPv4 / IPv6 and Python urllib2 force IPv4

I decided to do an DNS lookup and just send a Host header with the FQDN to grab the content. (Host headers are needed for vhosts)

Here is the ugly snippet:

# Ugly hack to get either IPv4 or IPv6 response from server
parsed_uri = urlparse(server)
fqdn = "{uri.netloc}".format(uri=parsed_uri)
scheme = "{uri.scheme}".format(uri=parsed_uri)
path = "{uri.path}".format(uri=parsed_uri)

try:
    ipVersion = ip_kind(fqdn[1:-1])
    ip = fqdn
except ValueError:
    addrs = socket.getaddrinfo(fqdn, 80)
    if haveIPv6:
        ipv6_addrs = [addr[4][0] for addr in addrs if addr[0] == socket.AF_INET6]
        ip = "[" + ipv6_addrs[0] + "]"
    else:
        ipv4_addrs = [addr[4][0] for addr in addrs if addr[0] == socket.AF_INET]
        ip = ipv4_addrs[0]

server = "{}://{}{}".format(scheme, ip, path)

url = urllib.Request(server, None, {'User-agent' : 'Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5'})
# Next line adds the host header
url.host = fqdn
content = urllib.urlopen(url).read()

This is far from ideal and it could be much cleaner but it works for me.

It is implemented here: https://github.com/SteveClement/ipgetter/tree/IPv6 This simply goes through a list of servers that return you your border gateway ip, now in IPv6 too.