python 3.6 installation and lib64

Question

I'm trying to install the new Python (3.6, released just a few weeks back) from source on openSUSE 42.2. Everything

./configure --prefix /home/paul/2017 --enable-optimizations
make
make install

appears to work just fine, but when I call the shiny new interpreter I get this error:

Python 3.6.0 (default, Jan  9 2017, 22:01:27)                                                                                                                                     
[GCC 4.8.5] on linux                                                                                                                                                              
Type "help", "copyright", "credits" or "license" for more information.                                                                                                            
Traceback (most recent call last):
  File "/home/paul/.pythonrc", line 7, in <module>
    import readline
ModuleNotFoundError: No module named 'readline'

Now, the module is there, it is installed under $PREFIX/lib64/python3.6/lib-dynload/readline.cpython-36m-x86_64-linux-gnu.so. Only the interpreter is not looking there (it is completely ignoring lib64, I checked with -vv).

Update: To be clear, this does not only affect readline but everything under lib64, most notably the modules under lib-dynload.

This appears to be a known issue, see the Python bugtracker. It's a fairly lengthy exchange ending in

Is there agreement on what needs doing here? I'd like to see this into 3.5 before it reaches its 10th birthday :)

This leaves me a bit confused. Is this supposed to be resolved? I do not remember ever running into this issue before, so I wonder if it is a 3.6 regression.

And obviously, any fix would be most welcome.

Update: for the time being I can work around the issue by symlinking everything under lib64 into lib. Obviously, that is not an ideal solution; at the very least it defeats the purpose of introducing lib64 (being able to have 32 and 64 bit versions side-by-side) in the first place. I'd much prefer something in a config file.

Answer 1

The opensuse openSUSE:Factory python3x has patches for that.

Links: Python 3.6 Python 3.8 Python 3.9

Spec files are usually a good starting point.

Note: the install step would install a python3 in /usr/local/bin. Changing that to altinstall avoids shadowing the system-wide python3.

Install

Create a downloads variable pointing to a directory where all downloaded files (python source and factory patches) will be saved

export downloads=/usr/local/build/downloads

In a user writable directory:

tar --no-same-owner -xvf $downloads/Python-3.6.4.tar.xz
cd Python-3.6.4

patch -p1 < $downloads/python-3.6.0-multilib-new.patch
patch -p0  < $downloads/python-3.3.0b1-localpath.patch
patch -p0 < $downloads/python-3.3.0b1-curses-panel.patch

# mind this step, otherwise
# none of the modules in `lib-dynload` could be imported !
autoreconf -i

mkdir build; cd $_
../configure --enable-optimizations --enable-ipv6 \
             --with-fpectl --enable-shared --with-system-ffi \
             --with-system-expat --enable-loadable-sqlite-extensions
make -j $(nproc)
# altinstall, not install (see above)
sudo make altinstall

Uninstall

To get rid of the installed files:

sudo rm -rf /usr/local/lib64/python3.6/
sudo rm -f /usr/local/lib64/libpython3.6m*
sudo rm -f /usr/local/lib64/libpython3.so
sudo rm -f /usr/local/lib64/pkgconfig/python-3.6*
sudo rm -f /usr/local/bin/python-3.6*

For python3.7

tar --no-same-owner -xvf $downloads/Python-3.7.3.tar.xz
cd Python-3.7.3
patch -p1 < $downloads/python-3.7.3-python-3.6.0-multilib.patch
patch -p1 < $downloads/python-3.7.3-distutils-reproducible-compile.patch
patch -p0 < $downloads/python-3.7.3-python-3.3.0b1-localpath.patch
patch -p0 < $downloads/python-3.7.3-00251-change-user-install-location.patch

# do not forget !
autoreconf -i

mkdir build; cd $_
../configure --enable-optimizations --enable-ipv6 --enable-shared --with-system-ffi --with-system-expat --enable-loadable-sqlite-extensions

make -j $(nproc)

sudo make altinstall

For python 3.8

tar --no-same-owner -xvf $downloads/Python-3.8.1.tar.xz
cd Python-3.8.1/
patch -p1 < $downloads/python-3.8.1-F00102-lib64.patch
patch -p1 < $downloads/python-3.8.1-F00251-change-user-install-location.patch
patch -p1 < $downloads/python-3.8.1-SUSE-FEDORA-multilib.patch
patch -p1 < $downloads/python-3.8.1-distutils-reproducible-compile.patch
patch -p1 < $downloads/python-3.8.1-python-3.3.0b1-localpath.patch

# do not forget !
autoreconf -i

mkdir build; cd $_
../configure --enable-optimizations --enable-ipv6 --enable-shared \
             --with-system-ffi --with-system-expat \
             --enable-loadable-sqlite-extensions
make -j $(nproc)
# altinstall, not install (see above)
sudo make altinstall