I'm writing research ML code and for reproducability I want to keep track of the code's version when each experiment is ran, similar to how nightly builds of various software track their version. I structure my code as a pip package. In setup.py I had a function get_hash using git-python to check the git hash at build time and insert it as __version__ of the installed package, which can be later saved a training log file.
Now I'm trying to modernise the build system and use pyproject.toml/setup.cfg definition. It does allow for dynamic versioning but only by capturing an attribute from the package built. This means I need to put get_hash in the package itself, which in turn makes all my package dependencies become build dependencies as well (since the package needs to be imported at build time). This is bad because I want to build wheels on my laptop, without the heavy-weight GPU-enabled dependencies.
I found a hackaround by creating what is technically a second dummy package, called my_package_version, which consists solely of __init__.py and uses git-python to set its __version__. Then in pyproject.toml I can write
[tool.setuptools.dynamic]
version = {attr = "my_package_version.__version__"}
This does the job, but feels very hacky and I end up installing my_package and a "ghost" my_package_version, which could be confusing to some users. What would be the best way to solve this issue?
TL;DR: use or re-implement setuptools_scm.
[project]
name = "my_package"
dynamic = ["version"]
[build-system]
requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
[tool.setuptools_scm]
# Empty is fine
Implementation from scratch requires moving the version code to a package hooking entry points for dynamic metadata in the same way:
[project.entry-points."setuptools.finalize_distribution_options"]
setuptools_scm = "setuptools_scm._integration.setuptools:infer_version"
You'll get called for every package you install, so care is required to avoid overriding your dependencies' versions. I recommend reading the code for setuptools_scm and its infer_version entry point in particular.
If it all seems a bit much, PyPA already did the work.