I get the following error when i try to implement some kind of dynamic class loading using __import__ ...:
No module named pip._vendor.requests.status_codes
Traceback (most recent call last):
File"/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~ticatestdev/2.386059027148196174/tica/tasks.py", line 36, in post
m = __import__(handler['module'], fromlist=[handler['name']])
File "/base/data/home/apps/s~ticatestdev/2.386059027148196174/tica/sources/processors.py", line 18, in <module>
from pip._vendor.requests.status_codes import codes
ImportError: No module named pip._vendor.requests.status_codes
... or importlib.import_module :
No module named pip._vendor.requests.status_codes
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~ticatestdev/2.386058813285719320/tica/tasks.py", line 36, in post
m = importlib.import_module(handler['module'])
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/base/data/home/apps/s~ticatestdev/2.386058813285719320/tica/sources/processors.py", line 18, in <module>
from pip._vendor.requests.status_codes import codes
ImportError: No module named pip._vendor.requests.status_codes
I import importlib of course.
I deploy my program on the Google AppEngine server to test it (I have a production URL and a Development URL).
Here is the code :
import importlib
import json
import logging
class Process(handlers.BaseHandler):
def post(self):
parameters = json.loads(self.request.get('parameters'))
if parameters:
if 'handler' in parameters:
handler = parameters['handler']
m = __import__(handler['module'], fromlist=[handler['name']])
#m = importlib.import_module(handler['module'])
task_handler = getattr(m, handler['name'])(parameters)
#task_handler = getattr(m, handler['name'])(parameters)
task_handler.startProcessing(parameters)
else:
logging.error("python.tasks.Process(): No handler information provided!")
else:
logging.error("python.tasks.Process(): No parameters provided!")
and the parameters contents:
{"dictionary_list": [...], "handler": {"name": "SourceProcessingHandler", "module": "sources.processors"}, "command": "process", "fsm": {}, "language_code": "ja"}
I could not find a similar error on google,
I wonder now if I am not wasting my time trying to outsmart myself with this complex solution ...
May be I should use a simple dictionary associating a name to the python classes?
The tica/sources/processors.py file appears to be part of your application. If so it explicitly depends on the pip package, which doesn't appear to be part of the standard python library and is not included in the GAE Runtime-Provided Libraries. This means it would need to be vendored in.
All the packages/modules that you want to dynamically load together with all their dependencies may similarly need to be vendored in.
I'd also add a lot more sanity checks before attempting to execute the dynamically loaded code. At least catching and handling ImportError.
Side note: just in case task_handler.startProcessing(parameters) can take too long and cause DeadlineExceededError for the post request you may want to delegate the execution onto a task queue or a backend execution module instead.