Search code examples
pythonpython-import

How to make `import` statements to import from another path?


I want to "hack" python's import so that it will first search in the path I specified and fallback to the original if not found.

The traditional way of using sys.path will not work if there is a __init__.py, see below.


CASE 1: The following works:

Files:

.
├── a
│   ├── b.py       # content: x='x'
│   └── c.py       # content: y='y'
├── hack
│   └── a
│       └── b.py   # content: x='hacked'
└── test.py        # content: see below
# test.py
import sys
sys.path.insert(0, 'hack') # insert the hack path

from a.b import x
from a.c import y

print(x, y)

Running test.py gives hacked y as desired, where x is hacked :)


CASE 2: However, if there is a __init__.py in a, it will not work.

Files:

.
├── a
│   ├── b.py
│   ├── c.py
│   └── __init__.py # <- NOTE THIS
├── hack
│   └── a
│       └── b.py
└── test.py

Running test.py gives x y, where x is not hacked :(


CASE 3: To fix case 2, I tried adding __init__.py to the hack path, but this disables the fallback behavior.

Files:

.
├── a
│   ├── b.py
│   ├── c.py
│   └── __init__.py
├── hack
│   └── a
│       ├── b.py
│       └── __init__.py # <- NOTE THIS
└── test.py

Running test.py raises the following error as there is no c.py in the hack path and it fails to fallback to the original path.

ModuleNotFoundError: No module named 'a.c'

My question is, how to make case 2 work?

Additional background:

The above cases are just simplified examples. In the real situation,

  • Both a and hack/a are large repos with many subfolders and files.

  • The imported modules (e.g. a.b) may also contain import statements that need to be hacked.

Therefore, ideally the solution would be to only add a few lines of code at the top of test.py rather than modifying exisiting code.

UPDATE: I have come up with a solution below (I cannot accept my own answer in 2 days). If you have better solutions or suggestions, please feel free to discuss.


Solution

  • The solution is to overwrite the default __import__ function (which is used by import statements) so that it first tries to import from the hack folder.

    __import = __import__ # save the original
    
    def _import(name, *a, **b):
        try:
            return __import('hack.'+name, *a, **b)
        except ImportError:
            return __import(name, *a, **b)
    
    __builtins__.__import__ = _import # overwrite with our own