I'm trying to write a downloader wrapper module in python that may use pycurl, selenium or the good-ol' requests to download a URL, along the lines of
# import pycurl inside a try-except block, logging errors
# also import selenium stuff inside a try-except block logging errors
# then
def pycurl_downloader(*args, **kwargs):
raise NotImplementedError
if 'pycurl' in sys.modules:
def pycurl_downloader(url, char_encoding=None):
# actual implementation, now we are certain pycurl is there
# similar for selenium
then when a site doesn't make use of silly DOM-modifying JavaScript I could just use pycurl (or if it isn't found, fallback to requests gracefully), otherwise use the various selenium drivers (again with graceful fallback if a driver fails).
But flake8 complains of an F811 redefinition error (redefining pycurl_downloader and also selenium_downloader later too) in the code above.
I could try to use some _not_implemented_function dummy and assign the name pycurl_downloader to this, then write the _real_pycurl_downloader function and reassign:
pycurl_downloader = _not_implemented_func
if 'pycurl' in sys.modules:
def _real_pycurl_downloader(...):
# blah
pycurl_downloader = _real_pycurl_downloader
but that doesn't feel right. Does anyone have a better idea?
Using if
-else
to only define the function once should shut flake8 up:
if 'pycurl' in sys.modules:
def pycurl_downloader(url, char_encoding=None):
# actual implementation, now we are certain pycurl is there
...
else:
def pycurl_downloader(*args, **kwargs):
raise NotImplementedError