Search code examples
pythonpython-3.xpython-import

Python no module named X - absolute imports


Before I even start I want to say - I know, that there are like bazillion questions similar to this, but I couldn't find the answer to my problem. I have a dir structure like this:

.
├── project
│   ├── A
│   │   ├── __init__.py
│   │   └── somelib.py
│   ├── B
│   ├── C
│   │   └── C
│   │       ├── foo.py
│   │       └── __init__.py
│   └── __init__.py
└── run.sh

run.sh:

python3 project/C/C/foo.py

foo.py:

from project.A.somelib import somefunc


VS Code actually gets the intellisense in foo.py - it tells me what funcitons/variables I can import from somelib. But when I run run.sh, I get this error message:

from project.A.somelib import somefunc
ModuleNotFoundError: No module named 'project'

Is there a way to solve this while preserving this directory structure?


  • adding project/__init__.py changed nothing
  • the sys.path in foo.py looks like this:
['/home/dabljues/projects/project/project/C/C', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3.7/site-packages']

restrictions:

  • I can't modify neither sys.path in the files nor PYTHONPATH before running the scripts
  • I can't pip-install anything
  • I don't have sudo-access
  • I can't create a virtualenv, since the scripts are supposed to be downloadable and quickly executable

Solution

  • IDEs like VSCode or Pycharm make their own assumptions about a project, and will usually correctly link modules even if the interpreter that will ultimately run the code can't.

    The reason why project.A.somelib can't be found is visible in your sys.path output, which gives you the places where python will search for modules. Since '/home/dabljues/projects/project/project' is not included, there is no way for python to resolve it during runtime.


    A quick hack

    You can just add the path manually to sys.path, either in the source file by running import sys; sys.path.insert(0, '/home/dabljues/projects/project/project/') in foo.py before any other imports happen, or by running export PYTHONPATH="${PYTHONPATH}:/home/dabljues/projects/project/project/" in your shell before run.sh.


    Installing the project

    Since it looks like you're developing a library, you might as well use the mechanisms python offers to make libraries shareable and thereby fixing any import issues. Add a minimal setup.py to the project root (i.e. /home/dabljues/projects/project/project/setup.py):

    from setuptools import setup, find_packages
    
    
    setup(
        name='project',
        version='0.1.0',
        packages=find_packages('project'),
    )
    

    And install your project in editable mode:

    $ python3 -m pip install -e .
    

    This will put a link in your python3 executable's site-packages that points to the project root, which makes it accessible whenever you run anything with python3.


    Tests

    I included print(__name__) at the top of all python files to get some output.

    running run.sh without installing the package:

    $ sh run.sh 
    Traceback (most recent call last):
      File "project/C/C/foo.py", line 1, in <module>
        from project.A.somelib import somefunc
    ModuleNotFoundError: No module named 'project'
    

    after installing it

    $ sh run.sh 
    __main__
    project.A.somelib
    

    As you can see, project.C.C.foo is executed as a script, yet it finds all imports that start with project because project is installed.