I have this script to crawl a website and find the Items I need..
from socket import timeout
from urllib.request import Request, urlopen, URLError
import bs4,urllib.parse
def track(self):
for _object in _objects:
req = Request('http://example.com/item.php?id='+str(_object))
req.add_header('User-Agent',
'Mozilla 5.0')
_URL = urlopen(req).read()
soup = bs4.BeautifulSoup(_URL, "html.parser")
allResults = []
i = 1
for hit in soup.findAll('cite'):
if ("% Off" in hit.text):
allResults.append(str(i) + ". " + hit.text + " | Item => " + _object)
i += 1
if (len(allResults) == 0):
print("No result found for this item => " + _object)
else:
for element in allResults:
print(element)
I want to throw an exception, so when connection failed to the website, or for any other reason it couldn't reach the URL, it prints "Something happened wrong"
I know I have to use socket.timeout but where should I put it in the code ?
Wrap the urlopen call into a try: except call:
try:
_URL = urlopen(req).read()
except Exception as e:
print("Something happened wrong: {}".format(e))
# do something, eg: continue