Search code examples
pythonsetup.py

inconsistent python package installation location with prefix

I am installing python package with prefix, and getting a following error.

$ python3 setup.py install --prefix=$HOME/some_prefix
running install

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/user/some_prefix/lib64/python3.4/site-packages/

and your PYTHONPATH environment variable currently contains:

    ''

Surely i can export PYTHONPATH=/home/user/some_prefix/lib64/python3.4/site-packages:$PYTHONPATH. But given that the path has python version, I found a fancy way from How do I find the location of my Python site-packages directory?

$ python3 -c "import distutils.sysconfig as sc; print(sc.get_python_lib(prefix=\"$HOME/some_prefix\"));"
/home/user/some_prefix/lib/python3.4/site-packages

Unfortunately, they mismatch on lib part, apart from negligible trailing /. So it still fails.

$ python3 setup.py install --prefix=$HOME/some_prefix
running install

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/user/some_prefix/lib64/python3.4/site-packages/

and your PYTHONPATH environment variable currently contains:

    '/home/user/some_prefix/lib/python3.4/site-packages'

Now, how can I properly match it?

I checked the documentation, but I don't think there is a relevant arguments.

https://docs.python.org/3/distutils/apiref.html#module-distutils.sysconfig distutils.sysconfig.get_python_lib([plat_specific[, standard_lib[, prefix]]])¶

$ python3 -c "import distutils.sysconfig as sc; print(sc.get_python_lib(\"A\", \"B\", \"C\"));"
C/lib64/python3.4

Also, which one is right? is lib or lib64? I see more packages in /usr/lib.

$ ll /usr/lib/python3.4/site-packages/ | wc
     70     554    5408
$ ll /usr/lib64/python3.4/site-packages/ | wc
     11      82     764

I am using standard python comes from centos 7.

$ uname -a
Linux localhost 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
$ which python3
/usr/bin/python3
$ python3 --version
Python 3.4.8
Solution

  • Fedora / CentOS patches various Python module to alter the installation locations, see this ServerFault post on the difference between /usr/lib and /usr/lib64.

    Note that the /usr/lib64 path is only used for platform specific code, not for pure-python libraries. So the correct method to use is to set the plat_specific argument to get_python_lib() (the first one) to True:

    $ python3 -c "import distutils.sysconfig as sc; print(sc.get_python_lib(True, prefix=\"$HOME/some_prefix\"))"

    However, if your target prefix directory is for a separate Python installation without the Fedora / CentOS patches, then you should really use the Python binary installed in that prefix to handle the installation:

    $HOME/some_prefix/bin/python3 setup.py install

    It'll come with all the right system configuration, including an unpatched distutils and a sysconfig module with the right local $PREFIX value, and no PYTHONPATH warning will be issued.

    And if you wanted to isolate packages from the system installation, use a virtualenv and install with the environment-specific Python binary:

    $ virtualenv name_of_virtualenv_directory
$ name_of_virtualenv_directory/bin/python setup.py install