Search code examples
pythonsetuptools

Include file from root level in setup.py


I'am trying to build a python package using a bamboo pipeline within our working environment.

To trigger the build I have to include a file build.info.txt that contains the version number. This file is at the top level in the repository, thus the repo structure is the following

build.info.txt
setup.py
MANIFEST.in
README.md
package/
   __init__.py
   file1.py
   file2.py

The file build.info.txt has the version number

version=x.y.z

I would like to use this file to trigger the version number, thus in the __init__.py I included a simple parser

from pathlib import Path
path = Path(__file__).absolute().parent / "build.info.txt"
__version__ = open(path).readline()[0].split("=")[-1]

This works fine in local mode. However, when I create a package and push it on our pypi server I get the following error at runtime (i.e. pip install package works):

FileNotFoundError: [Errno 2] No such file or directory: '</PATH/TO/ENVIRONMENT>/python3.8/site-packages/build.info.txt'

I changed the behavior of setup.py (I use setuptools) including:

setup(**other_params, package_data={"": ["../build.info.txt"]}, include_package_data=True)

while MANIFEST.in looks like

include build.info.txt

The issue is that build.info.txt using pip install is located at top level:

</PATH/TO/ENVIRONMENT>/python3.8/site-packages/build.info.txt

this file is shared among several packages, while I would like to have the file in

</PATH/TO/ENVIRONMENT>/python3.8/site-packages/<PACKAGE>/build.info.txt

(then I have to manage the parser in __init__ in order to parse the right file, but this is straightforward)

How can I manage this?

Thank you


Solution

  • My recommendation is to use importlib.metadata to read the version of the project.

    There was an attempt to formalize and standardize the convention of setting __version__ in PEP 396, but it never reached full acceptance. And also nowadays this habit goes away, because the need for it isn't that pressing anymore, thanks to importlib.metadata. So you could remove __version__ entirely and 3rd party pieces of code that want to know the version number of your application or library can use importlib.metadata.version('MyProject') to read it.

    If you insist on having __version__ in your library, you can turn this around and do the following in your __init__.py:

    __version__ = importlib.metadata.version('MyProject')
    

    Asides:

    • If you insist on working with the file build-info.txt, then you could symlink it in the package, for example as package/build-info.txt (linking to build-info.txt), and __init__.py could read that file at run-time.

      • Or, as suggested in the comments, symlink build-info.txt to package/build_info.py, and in __init__.py: from build_info import version as __version__ (this assumes that the content of that file is also valid Python syntax).
    • You should not use __file__ to read package data resources, this is unreliable, use importlib.resources instead (it will do this in a much cleaner way, and if needed take care of extracting the files in case the package is zipped, etc.).

    • There are tools that can take care of all this version number issues, setuptools-scm comes to mind and might fit your needs.