I recently started using elementary OS. It's based on Ubuntu. During installation it installs python 3.6.
I installed python 3.8 manually by installing the following packages: python3.8, python3.8-dev, python3.8-minimal, python3.8-venv.
I also updated the link to the python binary with:
sudo ln -sf /usr/bin/python3.8 /usr/bin/python3
After this a couple of things stopped working. For example when I tried to execute a non-existing command it didn't print the error message that it cannot found the command, but it displayed a python stack trace. That one I solved with:
cd /usr/lib/python3/dist-packages
sudo ln -s apt_pkg.cpython-36m-x86_64-linux-gnu.so apt_pkg.so
sudo ln -s apt_inst.cpython-36m-x86_64-linux-gnu.so apt_inst.so
As you can see the python error was because it couldn't find the compiled binaries for the apt module.
So this issue was solved, but there are a couple of similar ones, and none of them can be solved this way, as the module binary is not compatible with python 3.8.
Is it possible to remove python 3.6 completely and override it with 3.8 so that the module binaries also get updated? Or can python 3.8 and 3.6 coexist? I would be fine with the /usr/bin/python3 link pointing to python 3.6 and I would manually execute /usr/bin/python3.8 or create a different alias or link for it. However when I print out the sys.path with /usr/bin/python3.8 I get this:
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/home/{username}/.local/lib/python3.8/site-packages
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
The trouble is with the last one. That's where the modules are with the 3.6 compatible so files. Can I somehow force python 3.8 to completely ignore the last module search path (without always stating sys.path.remove in my scripts) ?
I recommend you undo your symlink overwriting your systemwide version of Python 3. As you have found, replacing the python3 executable might create some problems, as it is used under the hood.
In general, you should leave your system python[3] installation alone for this reason and it is common to use virtual environments. This can be done as follows (assuming you are in some project directory):
python3.8 -m venv venv
source venv/bin/activate
The first command will create a virtual environment ('venv') in the directory venv. The second command will 'activate': now python (and in this case, python3 and python3.8) all refer to your original python3.8 in the context of this shell. You will have to repeat this if you launch a new shell.
This will also allow you to install packages using pip without cluttering your system installation. The use of virtual environments and pip is an incredibly common workflow in the Python development world.
In terms of shell and 'global' Python management, you can also use pyenv to manage your Python versions and what is available in the shell. pyenv is quite nice if you want to run a particular version of python, say 3.8.0 but not 3.8.1.