Search code examples
pippython-module

How to organize my python modules and packages


So I have written a module A and a dedicated test-file. This module can be used alone, but I wrote it to be used as a "base" for a second module I wrote. This second module absolutely needs the first one, and also has it's own test-file. Finaly, I wrote a third module that is based on the first two modules. In other words :

  • module A, can be used without B or C.
  • module B needs module A, and can be used without C
  • module C needs module B (and so also A)

My question is how do treat all these modules ? For example, should I make each module a package, and then import A in B, and B in C ? Or should I put them all in a single package ? Also, what do I do of all the test files (put them next to the module, or all in a single test-folder) ?

As of today, I am treating each one as a package, but it seems a bit heavy to have to install A and B for using C:

+ moduleA
    - moduleA.py
    - test_moduleA.py
+ moduleB
    - moduleB.py
    - test_moduleB.py
+ moduleC
    - moduleC.py
    - test_moduleC.py

So I was thinking about merging all like this :

+ moduleC
    - moduleA.py
    - moduleB.py
    - moduleC.py
    + tests
        - test_moduleA.py
        - test_moduleB.py
        - test_moduleC.py

Is that the pythonic way to wrap my module C (and all its components) ? Or should I nest the modules in subpackages (moduleC.moduleB.moduleA) ?

The goal of all this is to export to github-like platform, and eventually pip.


Solution

  • The modules seem to be independent so they should be developed separately: separate development directories, separate git repositories.

    For proper installation the modules should declare dependencies: B depends on A and C directly depends on B and indirectly (transitively via B) depends on A. Such dependencies should be declared in setup.py:

    In B:

    setup(
        …
        install_requires=[A]
        …
    )
    

    In C:

    setup(
        …
        install_requires=[B]
        …
    )
    

    No need to declare dependency on A as B when being installed brings A with it.

    This way when installing pip install A A will be installed alone; when installing pip install B B will be installed with A; when installing pip install C C will be installed with both A and B.