Search code examples
pythondjangocherrypyimporterror

How to debug ImportError (with sys.path being correct)


I serve django pages via CherryPy. Everything works when CherryPy is started in the foreground. When I daemonize CherryPy with

Daemonizer(cherrypy.engine).subscribe()

I get an ImportError. sys.path is exactly the same in both cases (daemonized and non-daemonized). How can I debug this, what else than sys.path affects python imports?

Additional Information

Traceback:

[02/Sep/2014:03:08:46] ENGINE ImproperlyConfigured('Error importing module plinth.modules.first_boot.middleware: "No module named middleware"',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1353, in communicate
    req.respond()
  File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 868, in respond
    self.server.gateway(self).respond()
  File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2267, in respond
    response = self.req.server.wsgi_app(self.env, self.start_response)
  File "/usr/lib/python2.7/dist-packages/cherrypy/_cptree.py", line 299, in call
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 187, in call
    self.load_middleware()
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 45, in load_middleware
    mw_class = import_by_path(middleware_path)
  File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 26, in import_by_path
    sys.exc_info()[2])
  File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 21, in import_by_path
    module = import_module(module_path)
  File "/usr/lib/python2.7/dist-packages/django/utils/importlib.py", line 40, in import_module
    import(name)
ImproperlyConfigured: Error importing module plinth.modules.first_boot.middleware: "No module named middleware"
  • The file to import: /home/fbx/code/plinth/plinth/modules/first_boot/middleware.py
  • The relevant sys.path entry (also present when the ImportError occurs): '/home/fbx/code/plinth'
  • The ImportError occurs in djangos import_module function in https://github.com/django/django/blob/master/django/utils/importlib.py.
  • The parameter name for import_module is "plinth.modules.first_boot.middleware"
  • the django MIDDLEWARE_CLASSES setting is 'plinth.modules.first_boot.middleware.FirstBootMiddleware'

One more note:
I run the daemonized server with python -m plinth in the directory /home/fbx/code/plinth.
When I start the daemonized server with /usr/bin/python /home/fbx/code/plinth/plinth/__main__.py everything works! In this case, sys.path has one additional entry: /home/fbx/code/plinth/plinth. But adding this path manually on startup doesn't fix the ImportError when run as python -m plinth.

I'm running this code: https://github.com/freedombox/Plinth/tree/0b5af376102f4210395c15b2366b96a6e56fefb2

update
Thanks @cyraxjoe,os.chdir() in combination with the module missing in __init__.py was the problem. For me this behavior is unexpected and I did not find a lot of useful information/documentation, so I set up a github repo to easier demonstrate the issue: https://github.com/fonfon/ImportError-demo


Solution

  • This is just a theory, but it could be the reason.

    Given that:

    1. The Deamonizer plugin changes the directory to the root os.chdir('/').
    2. The package plinth.modules.first_boot is explicitly importing first_boot and not the middleware on the __init__.py of the package.

    It could be that before the Daemonizer plugins changes the directory, the module plinth.modules.first_boot is imported but without the middleware, so when django y trying to dynamically import the module it just find the module on the import cache but it can't find the middleware because the path was relative and when the Daemonizer plugin changes the directory then it becomes inaccessible.

    Try importing the middleware module on the __init__ of the package.

    Basically add a from . import middleware on the __init__