Search code examples
pythonimportrelative-import

Multiple relative imports in python 2.7 packages

I understand there are many SO questions on relative imports. I will document the extent I have tried the solutions therein, but I am still unable to solve my problem.

I have the following directory structure. It's not mine by design but I'm game for modifying things as necessary (forked repo).

exp
  main_Exp.py
  kaffe
    __init__.py
    tensorflow
      __init__.py
      network_shape.py
  ResNet
    __init__.py    (*)
    ThreeDMM_shape.py

To run the model in this repo, I am to use

python main_Exp.py input_file.csv

Inside main_Exp.py:

sys.path.append('./kaffe')
sys.path.append('./ResNet')
from ThreeDMM_shape import ResNet_101 as resnet101_shape
from ThreeDMM_expr import ResNet_101 as resnet101_expr

Inside ResNet/ThreeDMM_shape.py:

sys.path.append('/home/usc/Desktop/Research/FG18/ExpNet_Code_Release/kaffe/tensorflow')
from network_shape import Network_Shape

Ok, so obviously I need to change this hard-coded absolute path. I'd like to do it the right way and not use my own specific path that I happened to install these files to.

So I try

from ..kaffee.tensorflow import Network_Shape

>>> ValueError: Attempted relative import in non-package

(1) I added __init__.py file in the ResNet folder (shown with the (*)) (2) I tried running the file as a module: python -m main_Exp input_file.csv (3) I also tried adding __init__.py to the top level folder (exp), though I believe doing so is nonsense. (4) Given that the first import was happening using the kaffe path that was appended to sys.path, I tried changing import to from .tensorflow.network_shape import Network_Shape

Same error after all steps.

So I'm not understanding the rules around relative imports and how to reference files in a sane way. I would really appreciate a pointer that helps me understand how to do this, and how to think about such imports in general!

Solution

  • exp is indeed not a package and won't be made to a package even if you add an init file to it, cause that won't magically add it to the paths python looks for packages. If you do add the __init__.py you can then run as python -m exp.main_Exp input_file.csv (from ../exp). This would make python recognize exp as a package and kaffe/ResNet as subpackages. You would need to change imports to from ResNet.ThreeDMM_shape import ResNet_101 as resnet101_shape etc.

    Edit in response to comment: Running from the parent dir using the m switch is the recommended way of running the script see for instance https://stackoverflow.com/a/23540051/281545 (that's for python 3 however it should still apply). If you want to avoid it (it would break hardcoded relative paths for one) you should add exp to sys path (once maybe is enough) then change the imports to absolute ones as in:

    # main_Exp.py
sys.path.append(os.path.abspath(os.path.dirname(__file__))) # the exp folder
from ResNet.ThreeDMM_shape import ResNet_101 as resnet101_shape
from ResNet.ThreeDMM_expr import ResNet_101 as resnet101_expr

# ResNet/ThreeDMM_shape.py
from kaffee.tensorflow import Network_Shape