How to organize my packages under some root package in PyPi?

Question

I have few packages which are more or less independent (see apack and bpack below). I would like to have them all available like this:

import mycompany.apack.somemodule
import mycompany.bpack.somemodule

Since they are independent I have separate setup.py for each of them to deploy them to PyPi and therefore different release cycle.

Here's a question: Is it possible to have subpackages of root package with separated release procedure (each subpackage has it's own setup.py)? How to achieve that?

Here's what I tried, but wasn't able to get it working. My current setup.py looks like that:

from distutils.core import setup
setup(
    name='mycompany-apack',
    version='0.1',
    packages=['mycompany.apack'],
    license='GPLv3')

These setup.py are creating following structure in dist-packages folder:

mycompany
  apack
    __init__.py
    somemodule.py
  bpack
    __init__.py
    somemodule.py

Now since my mycompany does not have __ init __.py (this is my guess) I'm getting following error:

>>> import mycompany.apack.somemodule
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named mycompany.apack.somemodule

Answer 1

Short answer

Use pkg_resources.declare_namespace from the setuptools project.

Long answer

You could use setuptools instead of distutils. It supports dependency management and namespaces,

For apack the project structure will look like this:

/apack/
 |- mycompany/
 |   |- __init__.py (1)
 |   `- apack/
 |       |- __init__.py
 |       `- submodule.py
 `- setup.py

For apack your setup.py will look like this:

from setuptools import find_packages
from setuptools import setup

setup(
    name='apack',
    version='0.1',
    packages=find_packages())

The __init__.py marked as (1) will look like this:

import pkg_resources
pkg_resources.declare_namespace(__name__)

bpack will look very similar, except with apack replaced with bpack.

Let's assume there is also a cpack, which depends on apack and bpack.

It will look similar, but the setup.py will look like this:

from setuptools import find_packages
from setuptools import setup

setup(
    name='cpack',
    version='0.1',
    packages=find_packages(),
    install_requires=['apack', 'bpack'])

If cpack is installed, apack and bpack will be installed as well. Because of the namespace declaration, no conflicts occur

$ virtualenv venv
$ source virtualenv/bin/activate
$ pip install cpack
$ python
>>> from mycompany.apack import submodule
>>> from mycompany.bpack import submodule
>>> from mycompany.cpack import submodule