Search code examples
pythonpip

Behavior change in pip wheel


I have a Python application that I am upgrading from Python 3.6 to 3.9 (some of our dependencies don't support 3.10 yet). For... complicated reasons we use a single source tree to build various different configurations with different subsets of files, so part of setup.py is devoted to figuring out which files aren't used and deleting them. With Python 3.6.9 and Pip 20.3.3, I can just run pip wheel --no-deps . and it works fine, as pip creates a working directory in $TMPDIR, copies the source tree, and only works from the copy. However, using Python 3.9.14 and Pip 22.0.4, it seems something has changed and Pip now operates on the current directory itself - meaning it deletes the "original" copies of files instead. Not a huge problem, I can just use Git to restore everything to the last commit (or manually create and work from a temporary directory), but it would be simpler to enable the old behavior. Nothing in the documentation for pip wheel says anything about using (or not using) a temporary directory, and I can't find anything about when or why this behavior might have changed. Is it possible to do make newer versions of Pip enable this behavior?


Solution

  • It turns out to have been due to changing behavior of Pip. Pip 21.1 introduced the --use-feature=in-tree-build option and Pip 21.3 made in-tree builds the default while introducing a temporary deprecated option --use-deprecated=out-of-tree-build as a temporary workaround to enable the old behavior; however this was changed from deprecated to removed in Pip 22.1b1.

    Options we're considering for our workflow:

    1. use the exclude_package_data argument to setup()
    2. change our automated build system to run git checkout -- . afterwards automatically
    3. change our automated build system to copy the necessary files to a temporary directory itself, basically moving the functionality to a higher stage up
    4. Figure out a better way to accomplish what we currently accomplish by omitting files

    Which might better for you, oh traveller on the dunes of time, I cannot say.