How to import script with relative path?

Question

I have the following directory structure

   project/
        bin/
            script
        stuff/
            __init__.py
            mainfile.py

Inside of script, I have the following to enable command line execution:

#!/usr/bin/env python
from stuff import mainfile

This works, but I would have expected needing to jump up one level...

from ..stuff import mainfile

or

from project.stuff import mainfile

What am I missing here?

Answer 1

Actually none of your examples should work out of the box.

Let's modify bin/script.py somewhat:

#! /usr/bin/env python

import sys
print sys.path

from stuff import mainfile

This should yield something like

['.../project/bin', ...]
Traceback (most recent call last):
  File "bin/script.py", line 6, in <module>
    from stuff import mainfile
ImportError: No module named stuff

Only the directory of the script (not the current directory) is added to sys.path automatically:

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter.* If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

Hence there's no stuff module on sys.path. I'm not sure about your environment setup, but this is the canonical result when no additional parameters are set up (e.g. PYTHONPATH).

Similarily,

from ..stuff import mainfile

will result in the classic ValueError: Attempted relative import in non-package. You are informed of the fact that you can only do relative imports relative to actual modules. Since inside script .. does not refer to a module (because the script itself is the top level module so to say), a relative import does not work here. In a manner of speaking from Python's perspective, there is no module above the script and thus .. does not refer to something tangible when used in the context of the script.

Note that this also means that it does not help to only make project and project/bin into modules themselves by dropping __init__.py marker files. Relative imports to a parent of the script are only possible if the parent of the script is actually something python has a concept of.

This is one of the reasons, why the -m command line switch exists making it possible to run a module from the command line. For example, given the above relative import

python -m project.bin.script

does the trick, but only if executed from the proper directory (projectdir/..).

That problem is even worse with

from project.stuff import mainfile

because the project directory is only automatically on sys.path when you start a script from the directory above project and do not specify a main script to run:

cd <projectdir>/..
python -m project.bin.script
# works
cd <projectdir>
python -m bin.script
# does not work, because `sys.path` starts with <projectdir>
# and it's `stuff.mainfile` now, not `project.stuff.mainfile`.

If you want to import modules from project in your script, fix up sys.path to your needs:

import sys
import os

sys.path.insert(0, os.path.dirname(sys.path[0]))
from stuff import mainfile