Search code examples
pippyenvpyenv-virtualenv

Pip within Pyenv-virtualenv installing into system on Big Sur


I have installed and been using Pyenv for some time now, utilizing the Pyenv Virtualenv plugin to create venvs and activating automatically by including a .python-version as per the plugin docs.

It seems to not always work as expected, though. Recently upgraded OSX to Big Sur.

Pip seems to be installing packages globally, rather than into the venv:

pyenv versions
  system
  2.7.17
  3.8.5
  3.9.5
* 3.9.5/envs/project_venv (set by /Volumes/path_to_my_project/.python-version)
  fulfill

So far, so good.

pyenv which pip
/Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/bin/pip

As I would expect.

Here's where I'm confounded:

Outside of the venv:

$ pyenv which pip
/Users/mikekilmer/.pyenv/versions/3.8.5/bin/pip
$ pip list -v
Package           Version   Location                                                      Installer
----------------- --------- ------------------------------------------------------------- ---------
# all the global pip packages...
defusedxml        0.7.1     /usr/local/lib/python3.9/site-packages                        pip
# No Django, which is good
idna              2.10      /usr/local/lib/python3.9/site-packages                        pip

Inside of the venv:

$ pip install django
# installation output...
$ pip list -v
# abridged
Django            3.2.5     /usr/local/lib/python3.9/site-packages                        pip
## That's not supposed to be in the global packages!!!

And of course Django is also in the global pip list.

But wait...

Look at this:

$ /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/bin/pip install django
# installation output...
$ /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/bin/pip list -v
asgiref    3.4.1   /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip
Django     3.2.5   /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip
pip        21.1.1  /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip
pytz       2021.1  /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip
setuptools 56.0.0  /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip
sqlparse   0.4.1   /Users/mikekilmer/.pyenv/versions/3.9.5/envs/project_venv/lib/python3.9/site-packages pip

Now it's installed to the correct place.

But that's not the way it's supposed to work, is it? It seems that somehow, pyenv thinks that it's using the venv pip, but isn't.

When I run which pyenv, it returns this function:

pyenv () {
    local command
    command="${1:-}"
    if [ "$#" -gt 0 ]
    then
        shift
    fi
    case "$command" in
        (activate | deactivate | rehash | shell) eval "$(pyenv "sh-$command" "$@")" ;;
        (*) command pyenv "$command" "$@" ;;
    esac
}

I am not finding that code in the pyenv repository and I'm using the latest master (pyenv 2.0.3) via Homebrew.

Any insights?

Update

Seems that pip is symlinked to /usr/local/bin/pip3:

$ which pip
pip: aliased to /usr/local/bin/pip3

While pip3 points correctly to the pyenv location:

$ which pip3
/Users/mikekilmer/.pyenv/shims/pip3

$PATH starts like this:

/var/folders/62/38l28z6j12s9lqp6nnhmrlhc0000gn/T//zsh-501/bin:/usr/local/Cellar/pyenv-virtualenv/1.1.5/shims:/Users/mikekilmer/.pyenv/shims:/Users/mikekilmer/.pyenv/bin

The first directory is empty and seems to be part of the Darwin cache directory:

$ getconf DARWIN_USER_CACHE_DIR
/var/folders/62/38l28z6j12s9lqp6nnhmrlhc0000gn/C/

I'm not sure how it got appended to my PATH (by zsh?) or if it could somehow be making the shell think it should find pip elsewhere than with pyenv.


Solution

  • I was similarly confused at some point. Turned out that I installed / upgraded python and pip with different methods (brew, easy_install, curl, sudo ...). That's why my paths got messed up.


    Removing pip everywhere and reinstalling it helped.

    When I run which pyenv, it returns this function:

    pyenv () {
        local command
        command="${1:-}"
        if [ "$#" -gt 0 ]
        then
            shift
        fi
        case "$command" in
            (activate | deactivate | rehash | shell) eval "$(pyenv "sh-$command" "$@")" ;;
            (*) command pyenv "$command" "$@" ;;
        esac
    }
    

    You can add pyenv to the PATH in your ~/.zshrc or ~/.zshenv in order to get the correct path.

    echo 'export PATH=$HOME/.pyenv/bin:$PATH' >> ~/.zshenv