ModuleNotFoundError: No module named 'sharedFunctions'

I have a Python project in which I have the following folder structure:

> root
  > download_module
    > __init__.py
    > downloadProcess.py
    > sharedFunctions.py
    > someHelper.py
  > useSharedFunction.py

The download_module/__init__.py has the following code:

from .sharedFunctions import stringArgumentToDate
from .downloadProcess import downloadProcessMethod

The sharedFunctions.py file contains the following function:

def stringArgumentToDate(arg):
    dateformat = "%m/%d/%Y"
    date = None
    if arg.isnumeric():
        date = datetime.fromtimestamp(int(arg))
    if date == None:
        date = datetime.strptime(arg, dateformat)
    return date

Then on the useSharedFunction.py I try to import the shared function and use it like this.

from download_module import stringArgumentToDate
from download_module import downloadProcessMethod

def main():
    arg = '03/14/2022'
    dateArg = stringArgumentToDate(arg)

if __name__ == '__main__':
    main()

When I try to run this by using python3 useSharedFunction.py I got the following error:

Traceback (most recent call last):
File "useSharedFunction.py", line 4, in <module>
    from download_module import stringArgumentToDate
File "/Users/jacobo/Documents/project/download_module/__init__.py", line 2, in <module>
    from .download_module import downloadAndProcessMethod
File "/Users/jacobo/Documents/project/download_module/downloadProcess.py", line 10, in <module>
    from sharedFunctions import stringArgumentToDate, otherFunction
ModuleNotFoundError: No module named 'sharedFunctions'

I do believe the error is in downloadProcess since at the beggining of the file we got this import:

from sharedFunctions import stringArgumentToDate, otherFunction
from someHelper import Helper

Which refers to sibling files. However I'm unsure what will be a proper fix to allow to run the downloadProcess.py main independently but also, being able to call it one of its method from a root or any other file out of the module.

Solution

Consider this structure:

┬ module
|  ├ __init__.py
|  ├ importing_submodule.py
|  └ some_submodule.py
├ __main__.py
├ some_submodule.py
└ module_in_parent_dir.py

with content:

__main__.py
```
import module
```
/module/__init__.py
```
from . import importing_submodule
```
/module/importing_submodule.py
```
from some_submodule import SomeClass
```

/module/some_submodule.py

print("you imported from module")


class SomeClass:
    pass

/some_submodule.py

print("you imported from root")


class SomeClass:
    pass

/module_in_parent_dir.py
```
class SomeOtherClass:
    pass
```

How sibling import works

(skip this section if you know already)

Now lets run __main__.py and it will say "you imported from root".

But if we change code a bit..

/module/importing_submodule.py

from module.some_submodule import SomeClass

It now says "You imported from module" as we wanted, probably with scary red line in IDE saying "Unresolved reference" if you didn't config working directory in IDE.

How this happen is simple: script root(Current working directory) is decided by main script(first script that's running), and python uses namespaces.

Python's import system uses 2 import method, and for convenience let's call it absolute import and relative import.

Absolute import: Import from dir listed in sys.path and current working directory
Relative import: Import relative to the very script that called import

And what decide the behavior is whether we use . at start of module name or not.

Since we imported by from some_submodule without preceeding dot, python take it as 'Absolute import'(the term we decided earlier).

And then when we also specified module name like from module.some_submodule python looks for module in path list or in current working directory.

Of course, this is never a good idea; script root can change via calls like os.chdir() then submodules inside module folder may get lost.

Therefore, the best practices for sibling import is using relative import inside module folder.

/module/importing_submodule.py
```
from .some_submodule import SomeClass
```

Making script that work in both way

To make submodule import it's siblings when running as main script, yet still work as submodule when imported by other script, then use try - except and look for ImportError.

For importing_submodule.py as an example:

/module/importing_submodule.py

try:
    from .some_submodule import SomeClass
except ImportError:
    # attempted relative import with no known parent package
    # because this is running as main script, there's no parent package.
    from some_submodule import SomeClass

Importing modules from parent directory is a bit more tricky.

Since submodule is now main script, relative import to parent level directory doesn't work.

So we need to add the parent directory to sys.path, when the script is running as main script.

/module/importing_submodule.py

try:
    from .some_submodule import SomeClass
except ImportError:
    # attempted relative import with no known parent package
    # because this is running as main script, there's no parent package.
    from some_submodule import SomeClass

    # now since we don't have parent package, we just append the path.
    from sys import path
    import pathlib
    path.append(pathlib.Path(__file__).parent.parent.as_posix())

    print("Importing module_in_parent_dir from sys.path")
else:
    print("Importing module_in_parent_dir from working directory")

# Now either case we have parent directory of `module_in_parent_dir`
# in working dir or path, we can import it

# might need to suppress false IDE warning this case.
# noinspection PyUnresolvedReferences
from module_in_parent_dir import SomeOtherClass

Output:

"C:\Program Files\Python310\python.exe" .../module/importing_module.py

you imported from module
Importing module_in_parent_dir from sys.path

Process finished with exit code 0

"C:\Program Files\Python310\python.exe" .../__main__.py

you imported from module
Importing module_in_parent_dir from working directory

Process finished with exit code 0