How to move a module from root to a subpackage maintinaing backwards compatability

Question

This is the original structure of a project:

project/
  mypackage
    __init__.py
    moduleA.py

I want to move moduleA.py to a sub package:

project/
  mypackage
    __init__.py
    subpackage
      __init__.py
      moduleA.py

I need to maintain backward compatability so that old code using import mypackage.moduleA still works.

I tried adding this to project/mypackage/__init__.py:

from subpackage import moduleA

That allows me to import mypackage.moduleA. But unfortunately it forces moduleA to be imported as soon as mypackage is imported, which is not what I want (I am making moduleA an optional package, so dependencies are not guaranteed).

Can I still enable mypackage.moduleA imports using lazy module loading when import mypackage runs? The user should explicitly import mypackage.moduleA to trigger the import.

Answer 1

You could use a module level getattr hook to lazy load "moduleA" on first use.

# in mypackage/__init__.py

some_other_name = 123

def __getattr__(name):
    global moduleA
    if name == "moduleA":
        from mypackage.subpackage import moduleA
        return moduleA
    raise AttributeError(name)

Requires Python-3.7+. Note that the getattr is only invoked for names that aren't otherwise found in the namespace, so from mypackage import some_other_name will not invoke the hook and won't trigger an early import of the subpackage.

This will work:

from mypackage import moduleA

This will also work:

import mypackage
mypackage.moduleA

Though be aware of one important caveat to maintain backward compatibility. A direct submodule import will not work, because the submodule is not actually there to be found by the import system directly:

import mypackage.moduleA

To avoid breaking this form of import statement, you may still include a mypackage/moduleA.py file (which can be a deprecation shim)

# in mypackage/moduleA.py
import warnings
warnings.warn(
    message="mypackage.moduleA has moved to mypackage.subpackage.moduleA",
    category=DeprecationWarning,
    stacklevel=2,
)

from mypackage.subpackage.moduleA import *

The deprecation notice should hang around for a couple of releases, and then you can remove mypackage/moduleA.py entirely in the next breaking release, being sure to mention the change in your release notes.