Search code examples
pythonpython-wheel

How to build wheels for a project that requires other source projects?


I have a project A, that has source ("edit") dependencies on B and C (which happen to be in git submodules). I have a requirements file like this:

[requirements.txt]
nose
-e B/src
-e C/src

My setup.py lists these packages as deps:

[setup.py]
install_requires=[nose, B, C]

But when I install the wheel in another client project X, these "source" dependencies are not found (they are not part of A's wheel).

Collecting B (from A==0.0.1)
Could not find a version that satisfies the requirement B (from A==0.0.1) (from versions: )
No matching distribution found for B (from A==0.0.1)

Do I have to create separate binary distributions for each sub-project?


Solution

  • Dont put B and C to the install_requires list. That is for packages published via the cheese shop (ie pypi). You must put B and C under packages list.

    Remember that the dependencies in install_requires are not bundled up when you build your python release (binary or not) but packages in packages are bundled up with your release (you can view them as just "modules" rather than standalone libraries because that they would be).

    To explain this, if you have if you build your wheel and you give it to me and i try to install it, the nose package will be fetched from the cheese show and be installed. B and C will also be attempted to be installed this way but they fail because you never built them and release them to the pypi server.

    So you have 2 choices actually. First one is to build B and C individually and release each of them to the pypi server (or your own pypi server). Another option is to include B and C as packages on your setup.py file. This way, when you build the main project, both B and C are included in your release (ie the wheel file in your case).

    If you choose to build your local dependencies individually and upload them to the pypi server, then your setup should just work. Otherwise if you choose the other option, you have to move local dependencies to packages list. Here is the sample of a setup.py you might create in such a case:

    from setuptools import setup
    
    setup(
        name='myproject',
        version='1.0.0',
        author='myself',
        author_email='[email protected]',
        install_requires=['nose'],
        packages==['B', 'C']
        package_dir={'A': 'path/to/A/directory', 'B': 'path/to/B/dir'}
    )