Search code examples
pythonexceptionurllibconnection-timeout

Where and What exception I should use ? URLLib Python3


I have this script to crawl a website and find the Items I need..

from socket import timeout
from urllib.request import Request, urlopen, URLError
import bs4,urllib.parse
def track(self):
    for _object in _objects:
        req = Request('http://example.com/item.php?id='+str(_object))
        req.add_header('User-Agent',
                       'Mozilla 5.0')
        _URL = urlopen(req).read()
        soup = bs4.BeautifulSoup(_URL, "html.parser")
        allResults = []
        i = 1

        for hit in soup.findAll('cite'):
            if ("% Off" in hit.text):
                allResults.append(str(i) + ". " + hit.text + " | Item => " + _object)
                i += 1

        if (len(allResults) == 0):
            print("No result found for this item => " + _object)
        else:
            for element in allResults:
                print(element)

I want to throw an exception, so when connection failed to the website, or for any other reason it couldn't reach the URL, it prints "Something happened wrong"

I know I have to use socket.timeout but where should I put it in the code ?


Solution

  • Wrap the urlopen call into a try: except call:

    try: 
      _URL = urlopen(req).read()
    except Exception as e:
      print("Something happened wrong: {}".format(e))
      # do something, eg: continue