Is specific way of importing submodules at all possible?

Question

I am working in the following directory tree:

src/
    __init__.py
    train.py
    modules/
        __init__.py
        encoders/
            __init__.py
            rnn_encoder.py

My pwd is the top-level directory and my __init__.py files are all empty. I am executing train.py, which contains the following code snippet.

import modules

# RNNEncoder is a class in rnn_encoder.py
encoder = modules.encoders.rnn_encoder.RNNEncoder(**params)

When I execute train.py, I get an error saying that

AttributeError: module 'modules' has no attribute 'encoders'

I am wondering if there is any clean way to make this work. Note that I am not looking for alternative methods of importing, I am well-aware that this can be done in other ways. What I'd like to know is whether it is possible to keep the code in train.py as is while maintaining the given directory structure.

Answer 1

Putting an __init__.py file in a folder allows that folder to act as an import target, even when it's empty. The way you currently have things set up, the following should work:

from modules.encoders import rnn_encoder
encoder = rnn_encoder.RNNEncoder(**params)

Here, python treats modules.encoders as a filepath, essentially, and then tries to actually import the code inside rnn_encoder.

However, this won't work:

import modules
encoder = modules.encoders.rnn_encoder.RNNEncoder(**params)

The reason is that, when you do import modules, what python is doing behind the scenes is importing __init__.py from the modules folder, and nothing else. It doesn't run into an error, since __init__.py exists, but since it's empty it doesn't actually do much of anything.

You can put code in __init__.py to fill out your module's namespace and allow people to access that namespace from outside your module. To solve your problem, make the following changes:

modules/encoders/__init__.py

from . import rnn_encoder

modules/__init__.py

from . import encoders

This imports rnn_encoder and assigns it to the namespace of encoders, allowing you to import encoders and then access encoders.rnn_encoder. Same with modules.encoders, except a step upwards.