How would I suppress the following debugging error messages I'm getting from grequests
?
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
result = self._run(*self.args, **self.kwargs)
File "/usr/local/lib/python2.7/dist-packages/grequests.py", line 71, in send
self.url, **merged_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 384, in send
raise Timeout(e, request=request)
Timeout: HTTPSConnectionPool(host='itunes.apple.com', port=443): Read timed out.
<Greenlet at 0x7ff0c8165910: <bound method AsyncRequest.send of <grequests.AsyncRequest object at 0x7ff0c880b8d0>>(stream=False)> failed with Timeout
Here is what I've tried so far:
# disable requests/grequests logging
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("grequests").setLevel(logging.CRITICAL)
logging.getLogger("urllib3").setLevel(logging.CRITICAL)
logging.getLogger("gevent").setLevel(logging.CRITICAL)
The error occurs here:
rs = (grequests.get(url, headers=headers, cookies=cookies, proxies=proxies, timeout=timeout) for url in urls)
res_items = grequests.map(rs) # <-- this command produces all the errors
I am getting literally thousands of these messages, which I don't want to see. I am also using log.debug
in a lot of other places so don't want to change the debugging level program-wide. How would I suppress only the grequests
debugging messages?
For this you need to do three things:
(1) Make sure you have the most recent version of grequests
installed:
sudo pip install git+https://github.com/kennethreitz/grequests.git
(2) Have a handler for the grequests.map
function:
def exception_handler(request, exception):
pass # suppress errors on grequests.map
res_items = grequests.map(rs, exception_handler=exception_handler)
(3) Suppress the urllib3
warnings:
import urllib3
urllib3.disable_warnings()
logging.captureWarnings(True)
Now, there shouldn't be any more warnings coming from grequests
.