Search code examples
pythonpython-3.xurllibgoogle-custom-search

Connection Reset by Peer with Python module urllib


What I'm tryng to do is to download images through Google Custom Search API using Python. It's my first time and maybe I'm making some banal error catching exceptions with Python.

It all works fine even when I get into some error which is not 10054 Connection Reset By Peer. The code is something like this, I've just taken out the useless part:

try: 
    urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
except URLError as e:
    print(e.reason)

Sometimes it happens that connection is reset by peer and the console shows this error.

urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
File "C:\Python33\lib\urllib\request.py", line 210, in urlretrieve
   block = fp.read(bs)
File "C:\Python33\lib\http\client.py", line 503, in read
   return super(HTTPResponse, self).read(amt)
File "C:\Python33\lib\http\client.py", line 542, in readinto
   n = self.fp.readinto(b)
File "C:\Python33\lib\socket.py", line 297, in readinto
   return  self._sock.recv_into(b)
   ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
   Press any key to continue . . .

Now I'm not really interested in make that URL work, but I just want my loop to keep going and not to stop. How can I catch that exception?

Edit:

I also noticed that sometimes the code correctly catches the error and print(e.reason) correctly outputs [WinError 10054] without stopping the loop. That's very strange.


Solution

  • If you don't know the exact problem, you can catch all exceptions as so:

    try: 
        urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
    except URLError as e:
        print(e.reason)
    except KeyboardInterrupt as ki:
        raise ki
    except:
        print("Unknown Error")