Search code examples
pythonpython-modulepython-packaging

How importing packages and subpackages actually work?

I've been trying to understand how importing packages and subpackages works in Python. Based on what I understood from realpython.com, a package is simply a directory that contains python modules and a __init__.py file in which you could include an __all__ list object containing all the objects to import should you use an import * statement. On top of that, if you want to include more subpackages, each one should be a subdirectory with a __init__.py file in it.

So, let's assume I'm working on a project that looks something like this where my_pckg/ is a package I'm trying to create.

Project/
├── main.py
└── my_pckg/
    ├── __init__.py
    ├── functions.py
    └── abc/
        ├── __init__.py
        └── A.py

My understanding is that if I want to use a function func_1() from the functions module, I would have to run any of the following:

import my_pckg.functions as pckg_fun
pckg_fun.func_1()

from my_pckg.functions import func_1
func_1()

Equally, if I want to import a function func_A() from the A module of the abc subpackage, all I would have to do is use any of the following options:

import my_pckg.abc.A as pckg_A
pckg_A.func_A()

from my_pckg.abc.A import func_A
func_A()

Now, when I'm working with Python libraries like Numpy and I want to use the randn() function from the random subpackage, for example, I can simply do

import numpy as np
np.random.randn(2)

Whereas if I run the following snippet

import my_pckg as mp
mp.functions.func_1()

or

import my_pckg as mp
mp.abc.A.func_A()

I get an error like this one:

>>> mp.functions.func_1()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'my_pckg' has no attribute 'functions'

or this one, respectively

>>> mp.abc.A.func_A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'my_pckg' has no attribute 'abc'

So, I'm guessing that either there's more to creating modules that I thought, or I'm not understanding how modules like NumPy actually work.

Finally this joke by Mitch Hedberg truly summarizes the way I feel about all this:

They say the recipe for Sprite is lemon and lime, but I tried to make it at home, there's more to it than that. [...]

Solution

  • In __init__.py add import statements of the stuff you want to use from that module when you import only the module.

    Using your example, in my_pckg/__init__.py, you would have

    import functions
import abc.A

    After that, you should be able to do what you described with no problems.

    Edit: Best practices for import within packages are slightly different. Even with modules in the same folder, instead of directly importing, you should import from . like this:

    from . import functions
from abc import A

    The side effect to the second line is that you will directly call mp.A.func_A() instead of mp.abc.A.func_A()