Search code examples
pythoncookieshttp-redirecturlopenhttp-error

How do I set cookies using Python urlopen?


I am trying to fetch an html site using Python urlopen.
I am getting this error:

HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop

The code:

from urllib2 import Request
request = Request(url)
response = urlopen(request)

I understand that the server redirects to another URL and that it is looking for a cookie.
How do I set the cookie it is looking for so I can read the html?


Solution

  • Here's an example from Python documentation, adjusted to your code:

    import cookielib, urllib2
    cj = cookielib.CookieJar()
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    request = urllib2.Request(url)
    response = opener.open(request)