Search code examples
pythonanacondapython-3.8virtual-environmentanaconda3

How do I make sure conda's environment-specific paths are at the top of sys.path?


I have a conda environment, myenv. I realized that the Python interpreter in this environment tries to import packages from /Users/me/.local/lib/python3.8/site-packages before /usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages. I would have expected all the environment-specific paths to be appended to the beginning of sys.path. Is this expected behavior? In this case, I want to import conda-installed versions of numpy, scipy and numexpr since they use the Intel MKL backend.

(myenv) me$ which python
/usr/local/anaconda3/envs/myenv/bin/python
(myenv) me$ python
Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
>>> '/Users/me/.local/lib/python3.8/site-packages/numpy/__init__.py'
>>> import sys
>>> # /Users/me/dir* refer to directories that contain local modules I've pip installed
>>> print("\n".join(sys.path))

/usr/local/anaconda3/envs/myenv/lib/python38.zip
/usr/local/anaconda3/envs/myenv/lib/python3.8
/usr/local/anaconda3/envs/myenv/lib/python3.8/lib-dynload
/Users/me/.local/lib/python3.8/site-packages
/Users/me/dir1
/Users/me/dir2
/Users/me/dir3
/usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages

Other things that might be relevant:

  • I use conda and pyenv side-by-side so have conda's auto_activate_base setting set to false (as suggested here).
  • The Python interpreter declares that it's version 3.8.5, but conda info lists python version as 3.8.3.final.0.

I'm using conda version 4.9.2 on a MacOS Version 11.2.2.


Solution

  • Thanks to @merv's comment, I realized that the issue was that pip's user-site is prioritized above the conda environment's package directory. I resolved this as @merv suggested: by uninstalling all the packages in pip's user-site.

    me$ pyenv shell 3.8.1
    me$ pip freeze --user | grep -Eo "[[:alnum:]\-]+==[[:digit:]]{1,2}\.[[:digit:]]{1,2}(\.[[:digit:]]{1,2})?" | xargs pip uninstall -y
    me$ conda activate myenv
    (myenv) me$ python
    Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
    [Clang 10.0.0 ] :: Anaconda, Inc. on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import numpy as np
    >>> np.__file__
    /usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages/numpy/__init__.py