Importing modules from adjacent folders in Python

Question

I wanted to ask if there is a way to import modules/functions from a folder in a project that is adjacent to another folder.

For example lets say I have two files:

project/src/training.py
project/lib/functions.py

Now both these folders have the __init__.py file in them. If I wanted to import functions.py into training.py, it doesn't seem to detect. I'm trying to use from lib.functions import * .I know this works from the upper level of the folder structure, where I can call both files from a script, but is there a way to do it files in above/sideways folders?

Answer 1

Fundamentally, the best way of doing this depends on exactly how the two modules are related. There's two possibilities:

1. The modules are part of one cohesive unit that is intended to be used as a single whole, rather than as separate pieces. (This is the most common situation, and it should be your default if you're not sure.)
2. The functions.py module is a dependency for training.py but is intended to be used separately.

Cohesive unit

If the modules are one cohesive unit, this is not the standard way of structuring a project in Python.

If you need multiple modules in the same project, the standard way of structuring the folders is to include all the modules in a single package, like so:

project/
    trainingproject/
        __init__.py
        training.py
        functions.py
    otherfolder1/
        ...
    otherfolder2/
        ...

The __init__.py file causes Python to recognize the trainingproject/ directory as a single unit called a package. Using a package enables to use of relative imports:

training.py

# The . makes this relative, and it causes Python to search only
# in the current module's package for the module to import.
from . import functions

# The rest of training.py code

Assuming your current directory is project, you can then invoke training.py as a module:

python -m trainingproject.training

Separate units

If your modules are actually separate packages, then the simplest idiomatic solutions during development is to modify the PYTHONPATH environment variable:

sh-derviative syntax:

# All the extra PYTHONPATH references on the right ensure we don't
# lose an existing PYTHONPATH.
# Using $PWD ensures that the path in the environment variable is absolute.
PYTHONPATH=$PYTHONPATH${PYTHONPATH:+:}$PWD/lib/
python ./src/training.py

PowerShell syntax:

$env:PYTHONPATH =  $(if($env:PYTHONPATH) {$env:PYTHONPATH + ';'}) + (Resolve-Path ./lib)
python ./src/training.py

(This is possible in Command Prompt, too, but I'm omitting that since PowerShell is preferred.)

In your module, you would just do a normal import statement:

training.py

import functions

# Rest of training.py code

Doing this will work when you deploy your code to production as well if you copy all the files over and set up the correct paths, but you might want to consider putting functions.py in a wheel and then installing it with pip. That will eliminate the need to set up PYTHONPATH by installing functions.py to site-packages, which will make the import statement just work out of the box. That will also make it easier to distribute functions.py for use with other scripts independent of training.py. I'm not going to cover how to create a wheel here since that is beyond the scope of this question, but here's an introduction.