ModuleNotFoundError: No module named... importing from a sub-sub directory

I'm unable to import modules from a subfolder which imports a module from a sub-sub folder. I'm using python 3.6.

The structure of the folders look like this:

src
├── script.py
└── prepare_data
          ├── __init__.py
          ├── test.py      
          └── lib
               ├── aspect_extraction.py
               └── __init__.py

And in aspect_extraction.py I do:

def aspect_extraction():
    print("ok this worked")

test.py looks like this:

from lib.aspect_extraction import aspect_extraction
def test_func():
    aspect_extraction()

test_func()

script.py looks like this:

from prepare_data.test import test_func
test_func()

When I run pipenv run python src/script.py

File "/src/prepare_data/test.py", line 1, in <module>
from lib.aspect_extraction import aspect_extraction
ModuleNotFoundError: No module named 'lib'

Weirdly, when I run pipenv run python src/prepare_data/test.py, it works.

ok this worked

I cannot identify what the problem is... Is this related to python version?

Solution

import statement in test.py looks for lib.aspect_extraction in the current working dir, the one from where it was call (usually the one where script.py is, but not necesarily). Simplest solution (not the best): change the import statment:

from prepare_data.lib.aspect_extraction import aspect_extraction
def test_func():
    aspect_extraction()

test_func()

You can check the current working dir with os.getcwd(). That's your first reference for imports.

Also you can get the runing python file with os.path.dirname(os.path.abspath(file)). Insert these lines in script.py and test.py to check the difference:

import os
WORK_DIR = os.getcwd()
THIS_FILE_DIR = os.path.dirname(os.path.abspath(__file__))
print('WORK_DIR', WORK_DIR)
print('THIS_FILE_DIR', THIS_FILE_DIR)

Information about where python looks for modules:

"The first thing Python will do is look up the module in sys.modules. This is a cache of all modules that have been previously imported.

If the name isn’t found in the module cache, Python will proceed to search through a list of built-in modules. These are modules that come pre-installed with Python and can be found in the Python Standard Library. If the name still isn’t found in the built-in modules, Python then searches for it in a list of directories defined by sys.path. This list usually includes the current directory, which is searched first."

More in https://realpython.com/absolute-vs-relative-python-imports/

Hope this helps.