"_func" not being imported even when declared in __all__

Question

I am using Python 2.

I was able to find that __all__ by default does not pass _functions (underscored "internal" functions) when __all__ is not declared (i.e. just using from module import * but without __all__ explicitly defined). However, I'm also seeing that _functions are not passed even if they are added to __all__.

What words am I missing in my question to find the answer to this? Is this a continued "issue" (or expected behavior?) in Python 3?

My example is:

I have a mix of internal and external functions I am creating. Currently my bypass for this issue was to put the _functions (underscored functions, aka "internal functions", renamed without the underscore) into a ._internal_module_folder and then import the ._internal_module_folder into a external_module and add the external functions to the __all__ of external_module but leave out the internal functions.

So my original tree (with the issue) would look something like this:

modules_folder/
|---- __init__.py
|---- external_module.py
|---- _internal_module_folder/
|-------- __init__.py
|-------- _internal_module.py

where:

1. modules_folder/_internal_module_folder/__init__.py contains:

from ._internal_module import * # Should import _functions from __all__??

2. modules_folder/_internal_module_folder/internal_module.py contains:

__all__ = [
    # functions:
    'extl_func1',

    # _functions:
    '_intl_func1', 
    ]

def extl_func1(*args, **kwargs):
    pass
def _intl_func1(*args, **kwargs):
    pass

3. modules_folder/__init__.py contains:

from .external_module import *

4. modules_folder/external_module.py contains:

from ._internal_module_folder import *
__all__ = [
    # functions:
    'extl_func1', # from _internal_module_folder 
    'extl_func2',
    ]

def extl_func2(*args, **kwargs):
    _intl_func1()
def _intl_func2(*args, **kwargs):
    pass

When run I get an error that _int_func1 does not exist even though it's in __all__ explicitly:

NameError: name '_intl_func1' is not defined

My solution:

1. I renamed _intl_func to intl_func
2. I changed the import in modules_folder/__init__.py to

from . import _internal_module as im

3. I then aliased the internal functions on call

from . import _internal_module_folder as im
__all__ = [
    # functions:
    'extl_func1', # from _internal_module_folder 
    'extl_func2',
    ]

# Aliasing the internal module functions to pass to __all__:
extl_func1 = im.extl_func1

def extl_func2(*args, **kwargs):
    im.intl_func1()
def _intl_func2(*args, **kwargs):
    pass

Is this PEP 8 approved?

Summary: Put _function into __all__ but it wasn't passed on from module import *.

Answer 1

This is expected behavior. Internal functions, preceded by an underscore, are given a "weak internal use" indicator, which means they are not passed through __all__, even if explicitly named in __all__.

This information from PEP 8 (https://peps.python.org/pep-0008/#descriptive-naming-styles)

The key part of the solution was changing from importing all, i.e.:

from ._internal_module_folder import *

to importing the module, with an alias (can also be done without an alias!):

from . import _internal_module_folder as im # with module aliasing

Note: When importing the module, it is unnecessary to rename _intl_func1 to intl_func1, as importing the module bypasses __all__.

However, passing an external function (a function without a preceding underscore) from _internal_module_folder to the __all__ list in external_module.py is not the preferred method of exposing the external functions, as it can be difficult to find where the error is coming from if a function name is later changed! Imports should be controlled by __init__.py (per "Dead Simple Python", 2023, Jason C. McDonald, pages 87-88)

Therefore, modules_folder/__init__.py should be changed to include both modules' external functions:

from .external_module import *
from ._internal_module_folder import *

In this case modules_folder/external_module .py should be changed to:

from . import _internal_module_folder as im
__all__ = [
    # functions:
    'extl_func2', # Only list the external functions from THIS file
    ]

def extl_func2(*args, **kwargs):
    im.intl_func1
def _intl_func2(*args, **kwargs):
    pass

Alternatively, the specific internal function can be imported directly in modules_folder/external_module .py, simplifying the _intl_func1 function call:

from ._internal_module_folder import _intl_func1 # This does not access the __all__ list!
__all__ = [
    # functions:
    'extl_func2',
    ]

def extl_func2(*args, **kwargs):
    _intl_func1()
def _intl_func2(*args, **kwargs):
    pass