Search code examples
pythonheaderurllib2

Remove HTTP "Connection" header, Python urllib2


I'm trying to learn Python. At the moment I'm stuck with a simple thing :( When I send request with urllib2, it add "Connection:close" header. What should I do to send request WITHOUT this header?

My code:

request = urllib2.Request('http://example.com')
request.add_header('User-Agent', self.client_list[self.client])
opener = urllib2.build_opener()
opener.open(request).read()

Thanks guys, I'm very appreciate your help!


Solution

  • You can't. Not with urllib2 in any case. From the source code:

    # We want to make an HTTP/1.1 request, but the addinfourl
    # class isn't prepared to deal with a persistent connection.
    # It will try to read all remaining data from the socket,
    # which will block while the server waits for the next request.
    # So make sure the connection gets closed after the (only)
    # request.
    headers["Connection"] = "close"
    

    Use the requests library instead, it supports connection keep-alive:

    >>> import requests
    >>> import pprint
    >>> r = requests.get('http://httpbin.org/get')
    >>> pprint.pprint(r.json)
    {u'args': {},
     u'headers': {u'Accept': u'*/*',
                  u'Accept-Encoding': u'gzip, deflate, compress',
                  u'Connection': u'keep-alive',
                  u'Content-Length': u'',
                  u'Content-Type': u'',
                  u'Host': u'httpbin.org',
                  u'User-Agent': u'python-requests/0.14.1 CPython/2.6.8 Darwin/11.4.2'},
     u'origin': u'176.11.12.149',
     u'url': u'http://httpbin.org/get'}
    

    The above example uses http://httpbin.org to reflect the request headers; as you can see requests used a Connection: keep-alive header.