Search code examples
pythonpip

Why is Python leaving non-existent files in my package?


For whatever reason, when I install my package pie with pip install --break-system-packages --user git+https://github.com/zurgeg/pie.git@ed00801, it fails due to a mysterious phantom file:

Traceback (most recent call last):
  File "/home/jg/.local/bin/pie", line 5, in <module>
    from pie.pie import cli
  File "/home/jg/.local/lib/python3.11/site-packages/pie/pie.py", line 20, in <module>
    from .crusts.notdate.play.playdate_pie import playdate as crust_playdate
  File "/home/jg/.local/lib/python3.11/site-packages/pie/crusts/__init__.py", line 7, in <module>
    globals()[submodule] = _import_module("." + submodule, __name__)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jg/.local/lib/python3.11/site-packages/pie/crusts/notdate/__init__.py", line 5, in <module>
    if not submodule.startswith("_"):

Now, this file did exist at one point:

git -P log --all --full-history --no-color -- src/pie/crusts/notdate/__init__.py |
awk 'match($0, /commit/){ print substr($0, RSTART + 7, 7) }'
aaef1b9
33210c5
bd3ded3

That is, until I realized that the file was in the wrong place and moved it to its current position:

git diff aaef1b9 aaef1b9~1
diff --git a/src/pie/crusts/__init__.py b/src/pie/crusts/notdate/__init__.py
similarity index 58%
rename from src/pie/crusts/__init__.py
rename to src/pie/crusts/notdate/__init__.py
[...]

Deleting the file didn't do anything, Python still thinks it's there, and clearly it is, even after a pip uninstall pie. Now, I can just rm the file, but that's a temporary solution, because if I recreate it in the repository, install, and then delete it, the file is kept. Bumping the version to 0.2.2 doesn't work either.

I expect the file to just be deleted when I reinstall my package, but for whatever reason, it seems like pip isn't tracking that file as part of the package?

UPDATE: It still seems to be failing with all of these commands using both a venv and the system install:

  • pip install --no-cache-dir
  • pip install --force
  • pip install --force --no-cache-dir

Solution

  • TL;DR: Remove your build directory from your package directory

    I decided to make a dummy file that would raise an exception for debugging purposes:

    # Create a directory to put temporary debugging files
    # Pie loads everything under "crusts" automatically
    mkdir src/pie/crusts/temp
    echo "raise Exception" > src/pie/crusts/temp/__init__py
    

    Then, I uninstalled the package everywhere:

    pip uninstall --break-system-packages pie
    pipx uninstall pie
    

    From there, I made a clean venv:

    virtualenv ~/.venv
    

    Lastly, I installed pie in the venv, and checked it, and it failed, so I uninstalled it...

    # (from the venv)
    pip install ./pie
    pie
    # traceback
    Exception
    pip uninstall pie
    

    Lastly, I decided to see where the file remained, even with everything related to Pie uninstalled:

    echo **/temp/__init__.py
    pie/build/lib/pie/crusts/temp/__init__.py
    

    I then just deleted the build directory:

    rm -rf pie/build
    

    And then bam, it works!

    pie
    [INFO] pie: Loading crusts...
    # ...