Search code examples
pythonsetuptoolspython-packagingpython-wheelpyproject.toml

Include or exclude (license) files from package data with pyproject.toml and setuptools


TL;DR

How does one reliably include files from LICENSES/ (REUSE-style) in source archive and wheels for a Python package with a src/ layout? How does one exclude specific files?

Details

I have a project structure that looks like

.
├── pyproject.toml
├── LICENSES
│   ├── MAIN.txt
│   ├── SECUNDARY.txt
├── MANIFEST.in
├── random_package
│   ├── __init__.py
│   ├── foo1.cpp
│   ├── foo2.cpp
│   ├── submodule1
│   │   ├── __init__.py
│   │   ├── bar1.cpp
│   ├── submodule2
│   │   ├── __init__.py
│   │   ├── bar2.cpp

The pyproject.toml looks like

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "random_package"
version = "0.1.0"
license = {file = "LICENSES/MAIN.txt"}

[metadata]                          # EDIT: metadata was the issue
license-files = ["LICENSES/*.txt"]  # this line should be in [tool.setuptools]

[tool.setuptools]
package-dir = {"" = "."}
include-package-data = true  # tried both true and false

[tool.setuptools.packages.find]
where = ["."]
include = ["random_package*"]

How do I include all cpp files except submodule1/bar1.cpp into the installation?

I have tried the following entries in the toml (one at a time):

[tool.setuptools.exclude-package-data]
"*" = ["bar1.cpp"]
"random_package.submodule1" = ["bar1.cpp"]

I even set include-package-data to false and entered cpp files manually (except bar1.cpp) and even that did not work for both source and wheels.

Nothing works reliably: for any and all combinations of these options, I always get bar1.cpp in either the zip/tar.gz archive or the wheel when I do python -m build.

As for the license files, I get LICENSE/MAIN.txt in the source build, but not the others and no licenses are present in the wheels.

Partial solution

I have something that works for source dist using a MANIFEST.in with an include for the LICENSES/*.txt files and a manual include for the .cpp files instead of the data options in pyproject.toml but even this does not work for the wheel: I don't get the licenses in random_package-0.1.0.dist-info.

Am I wrong in expecting the license files in the wheel? With the old setup.py scheme, back when I was using a single License.txt file, I did get the license file in there... And is there no way to do that with the toml alone?


Solution

  • It turns out that I was mistaken about the location of license-files (I first saw it in the "metadata" section on the doc); it must actually be in [tool.setuptools]. The other data include issue was maybe a cache issue, it seems to work in the following pyproject.toml:

    [build-system]
    requires = ["setuptools>=61.0"]
    build-backend = "setuptools.build_meta"
    
    [project]
    name = "random_package"
    license = {file = "LICENSES/MAIN.txt"}
    version = "0.1.0"
    
    [tool.setuptools]
    package-dir = {"" = "."}
    include-package-data = false
    license-files = ["LICENSES/*.txt"]
    
    [tool.setuptools.packages.find]
    where = ["."]
    include = ["random_package*"]
    
    [tool.setuptools.package-data]
    random_package = ["*.cpp"]
    
    [tool.setuptools.exclude-package-data]
    "*" = ["bar1.cpp"]
    

    With this, no MANIFEST.in file is required.