Is there a clear, simple strategy to do relative imports in Python that doesn't require creating Packages, as I would like to use them between sub-folders of my project while I develop the project?
I'm referring to relative imports, like:
from ..mypackage import mymodule
from ..mypackage.mymodule import mymethod
from .. import mypackage
Only the most simple cases of relative imports actually work. Anything else results in the error message:
ImportError: attempted relative import with no known parent package
Creating a package makes little sense in this context, as I am simply in need of importing relatively between sub-folders in my main project. I use sys.path.append() as a workaround, but that should not be necessary, and it is not portable.
There is a SO post, Relative imports for the billionth time, which addresses this problem. The main article plus 15 answers and 36 comments indicate that this issue has been around for a long time. Many very smart people have offered exotic explanations and proposed cumbersome solutions to such a simple issue.
It seems the issue revolves around where Python thinks the importing module is in the file system at the time it attempts to execute the import statements. My opinion is that Python should be able to figure out if it is and exactly where in the file hierachy that is. I'm looking for a simple way to make import statements work as expected whether the importing module is called from another module, or if it is run from an interpreter, or an IDE like Pycharm.
Update: I wrote a package to solve this nasty issue, Sysappend:
https://pypi.org/project/sysappend/
Please refer to its documentation for the usage.
Some SO users may think that this question is "opinion-based" and try to close it. I think this is a valid question, because it asks of a clear programming issue with no clear solution. I will therefore give you my technical answer, by no means intended to be "the most appropriate solution".
I think that relative imports in Python are badly designed and best avoided, unless solutions are taken to make them intuitive1. I'm proposing two solutions below, by no means intended to be the "appropriate" or "pythonic" solution2.
If you are ok with an extra package, there are several targeting this issue. In a nutshell, these are just wrappers of sys.path.append, but making it more portable, and allowing "relative imports" to work as you would expect. Examples of such packages include importmonkey and ultraimport.
Taking a folder structure like:
root
├─ subfolder
│ └─ subscript.py
└─ rootscript.py
With rootscript.py:
def somerootfunction():
print("I'm a function at the root of the project")
You can use importmonkey in subscript.py to call somerootfunction():
from importmonkey import add_path; add_path("../")
from rootscript import somerootfunction
somerootfunction()
Not ideal, but it works. Linting and debugging are retained. I find it much simpler, effective, and easy to communicate than "the appropriate solutions"2,3.
Alternatively, in limited cases and after having considered a folder restructure or the other alternatives2, consider using symbolic links instead of relative imports.
You can link an entire parent folder placing a link in some sub folder, so Python can access it. Editing the linked code files by opening the items from the link will simply modify the source files. If you use a relative path during the link creation, this will also make the project portable (caveat: depends on the ported system interpretation of the link, see below). Finally, Git treats symbolic links as just another file that points to a location, so there is no issue with versioning.
The main downside is that this solution is OS-dependent (i.e. a symbolic link created on Linux won't be read as one if you open your project in Windows). There aren't workarounds to this, as far as I'm aware. It hasn't been an issue for my projects.
To create a symbolic link:
On Linux, I personally use vscode-symlink to make it easy from VSCode. Otherwise just use ln -s source_path destination_path.
On Windows, you can use mklink. If you're using it from a non-cmd terminal (e.g. the VSCode terminal) remember to prepend cmd /c before using it. E.g., if I'm in the sub-folder and I want to link to a parent folder file, cmd /c mklink rootscript.py ..\rootscript.py.
You can link to a directory with the /D argument.
1 The fact that there's lots of confusion on this matter is undeniable, which points to a design issue. This is proven by: A) the abundance of StackOverflow with no clear answer (e.g. Relative imports for the billionth time), Reddit posts and various website posts on the subject, many proposing conflicting solutions (of which I'm listing some examples in note 2 below). Further, this is proven by B) the existence of many dedicated packages targeting the issue, of which I'm listing some examples in my Solution 1.
2 I'd recommend going for solution 1 (use a dedicated package) before going for other solutions. Probably, "the appropriate" solution is to create a package and installing the project with a pyproject.toml file, and/or using flit/poetry. However, there is still a lot of reading and learning required to do it, and in the best case you still have to complicate your project with added files (e.g. __init.py__) or managing a package when you're still in development/debugging stage. Also, there is still confusion and disagreement on this solution, due to this having replaced older standard solutions. Others believe that pip install with editable is sufficient. Some suggest editing PYTHONPATH, which I think is downright bad.
I think that these solutions are viable but complicate life immensely, especially for newcomers. Some of these will actually work but introduce issues or complications with either debugging and/or linting the import source, making the programming experience far from ideal.
3 As a personal addendum, I believe that this is a strong indication of Python's bad design when it comes to relative imports, and another proof that Python is a good and easy scripting language, but a flawed, clumsy and hard to master programming language. For my large software projects I use other languages, and if I need Python (likely e.g. when dealing with Machine Learning), I use interop methods to call separate, small Python scripts.