Search code examples
pythonpython-3.xhttphttp-status-code-404urllib2

urllib2 catches 404 error while URL exists


I faced with strange bug: urllib2 catches 404 error, while openning a valid url. I tryed it in browser, the url can be opened. Also I pass user-agent.

import urllib.request as urllib2
uri = 'https://i.ytimg.com/vi/8Sii8G5CNvY/hqdefault.jpg?custom=true&w=196&h=110&stc=true&jpg444=true&jpgq=90&sp=68&sigh=OIIIAPOKNtx1OiZbAqdORlzl92g'
try:
  req = urllib2.Request(uri, headers={ 'User-Agent': 'Mozilla/5.0' })
  file = urllib2.urlopen(req)
except urllib2.HTTPError as err:
  if err.code == 404:
    return "Not Found"

Why I get this error? Thank you for answers.


Solution

  • If you want to get the body anyway, simply read the error response with an err.read():

    import urllib2
    uri = 'https://i.ytimg.com/vi/8Sii8G5CNvY/hqdefault.jpg?custom=true&w=196&h=110&stc=true&jpg444=true&jpgq=90&sp=68&sigh=OIIIAPOKNtx1OiZbAqdORlzl92g'
    try:
      req = urllib2.Request(uri, headers={ 'User-Agent': 'Mozilla/5.0' })
      file = urllib2.urlopen(req)
    except urllib2.HTTPError as err:
      if err.code == 404:
        print "Not Found"
        print err.read()