Search code examples
pythonvirtualenvpython-internals

Python and virtualenv - why is the python version used to name subdirectories?

Why does the virtualenv creation mechanism insist on naming subdirectories with the version of Python used? I am referring to something like test_venv/venv3/lib/>>>python3.6<<</site-packages

When you're in a 3.6 venv, you of course get 3.6 packages. And same with 2.7 meaning 2.7 packages. And you can't use these without activate-ing the venv beforehand so the risk of confusion seems low.

Among other things people often have questions about why something is not working that have to do with either the OS system path or the python sys.path. And having those subpaths named after the python version makes it hard to generalize on where to find site-packages.

edit: the following is on macOS, but I got roughly the same behavior on an Ubuntu 18.04 VM.

create a python 2 virtual environment with virtualenv ./venv2

directory structure with tree -d -L 3 ./venv2/:

./venv2/
├── bin
├── include
│   └── python2.7 -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
└── lib
    └── python2.7
        ├── config -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
        ├── distutils
        ├── encodings -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings
        ├── lib-dynload -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
        └── site-packages

create a python 3 virtual environment with python3 -m venv ./venv3

Slightly better, but still impacts site-packages.

./venv3/
├── bin
├── include
└── lib
    └── python3.6
        └── site-packages
Solution

  • It is consistent with the specification:

    Running this command creates the target directory... And it creates an (initially empty) lib/pythonX.Y/site-packages (or Lib\site-packages on Windows) subdirectory.

    This is the cpython bit that implements it if you were curious.

    As to why: it reflects (and as noted (and reflected in specification and code) different on Windows) behavior of how search paths are constructed from sys.prefix (which in turn would be determined from location of python executable in your venv in this case). Resolution thereof is implemented in Modules/getpath.c and also contains more detailed description in comments.

    Long story short: It allows venv to relatively easily use existing code for all the search path handling work it needs.