Packaging and building python namespace packages

Question

What I am trying to do?
Have one large package with several installable parts, which need to work together, but specific parts can be used on their own. I decided to use namespace package, as it seems to do what I want.

Problem
How do I actually build a namespace package?

My example
I have a following namespace package, with each of the subpackages containing some modules not listed here. I am using setuptools.

.
|-mypkg
  |- core
  |    |- setup.py
  |    |- mynmspc
  |         |- util
  |- compute
  |    |- setup.py
  |    |- mynmspc
  |         |- funcs
  |- plotting
       |- setup.py
       |- mynmspc
            |- graphs
- setup.py

Now I can build a wheel of this package python setup.py bdist_wheel. However if I want to have e.g. core subpackage installable by itself do I have to create another wheel for it? What would be the best practice if I would like to give users option to either install the whole package or any of the listed subpackages? Is there a way to automatically build all nested wheels of the main package?

Answer 1

If you want to use latest python native namespace packages to allow users to install extensions-like packages (mypkg-core, mypkg-compute,...), you should create different folders for each package like so

mypkg-core/
    mypkg/ # no __init__.py here
        core/
            __init__.py
        setup.py

mypkg-compute/
    mypkg/ # no __init__.py here
        compute/
            __init__.py
        setup.py

where each setup.py is like the following

from setuptools import setup, find_namespace_packages

setup(
    name='mypkg-core',
    ...
    packages=find_namespace_packages(include=['mypkg.*'])
)

And you will need to build a whl for each package.

With this configuration, users will be able to

pip install mypkg-core
pip install mypkg-compute

And access them via the same mypkg namespace:

import mypkg.core
import mypkg.compute

Don't forget to use find_namespace_package from setuptools in setup.py for each subpkg.

Please always refer to the official documentation about native namespace packages..

However, I would suggest an easier way to distribute your package by adding the subpackages dependencies as extras grouped with the name of the subpackage. This way the users can use the subpackage only if they install the whole package with

pip install mypkg[compute]