Search code examples
pythonvirtualenvpython-packagingpython-venvpython-poetry

How to build a self-contained Python venv?

Is that possible to build a self-contained Python virtual environment? By self-contained, I mean that all needed files to execute the program are there, including Python.

I'm using Poetry to manage venvs. Looking at venvs I created using poetry install I see that the dependencies are actually copied, but Python is simblinked.

For example:

>> ls -lah my-venv/bin/
python -> /Users/my-user/.pyenv/versions/3.11.2/bin/python

Also, I tried the virtualenvs.options.always-copy Poetry config, which translates to --always-copy virtualvenv configuration, but it didn't copy Python.

Since an expected answer would be "use containers", I say in advance that it's not an option for the given use case.

This question aims to a solution that an "all-in" directory/file can be uploaded to a Linux server and just run without using depending on any system-installed software.

Solution

  • The simplest solution I found:

    1. Use conda/mamba to manage the dependencies. It has the advantage of actually installing Python, which is useful for the question use case.
    2. Use conda-pack to pack in a venv every Python user knows how to use.

    Why this approach solves the problem? Because we end up with a venv including actual binaries, instead of symblinks. So we can't just share it as tar.gz file and no one (other than the project developers) has to learn new technologies (every average Python developer is likely to know venv). It just works:

    # ON THE SOURCE/CI MACHINE
$ conda pack -n my_env -o my_env.tar.gz

# ON THE USER/SERVER MACHINE
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
$ source my_env/bin/activate

    A note: I tried using like PyInstaller and other tools people wrote in answers and comments. Honestly, they try to reinvent the wheel. For example, PyInstaller goes recursively over the imports and tries to build from the discovered imports. On my first try it didn't work for PyArrow *.pyx dependencies, for example. Maybe I used it wrong. But, why invent the wheel and discover the imports if we can go explicit with major tools like standard venv, Poetry, and Conda/Mamba.