I'm using python3 via linuxbrew and today it seems there was a crucial python-update replacing python 3.7 with 3.8 altogether.
As for my OS (Lubuntu 18.04 LTS):
Operating System: Ubuntu 18.04.4 LTS
Kernel: Linux 5.3.0-62-generic
Architecture: x86-64
That's why the pip-upgrade of all my python site-packages didn't work then:
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
The error thrown was:
# /home/andylu/Desktop/Shell/python_package_complete_update_via_homebrew_n_pip: /home/linuxbrew/.linuxbrew/bin/pip: /home/linuxbrew/.linuxbrew/opt/python/bin/python3.7: bad interpreter: No such file or directory
# WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
# Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
# To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
# Defaulting to user installation because normal site-packages is not writeable
# ERROR: You must give at least one requirement to install (see "pip help install")
Later, I found out that there is a new python-version install. Weird enough, there were two folders created:
/home/linuxbrew/.linuxbrew/lib/python3.8/home/linuxbrew/.linuxbrew/Cellar/python@3.8/3.8.3_2/lib/python3.8As the site-packages in the new python3.8 folder were quite empty, I moved all the site-packages (> 1GB) from the remnant/residual python3.7 to the new python3.8 folder. With this, I assumed they'll function as well under python 3.8, instead of deleting and reinstalling them all.
Moreover, I figured that the site-packages in /home/linuxbrew/.linuxbrew/Cellar/python@3.8/3.8.3_2/lib/python3.8 seems to be a mere link to the folder under /home/linuxbrew/.linuxbrew/lib/python3.8.
I updated then the pip package-update command to the following:
python3 -m pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 python3 -m pip install -U
This seems to work. Yet, upon opening VS Code, it always states in the bottom-right "Path to the pylint linter is invalid" even though the path exists:
USER:~/Desktop/Python/Scripts$ which pylint
/home/linuxbrew/.linuxbrew/bin/pylint
USER:~/Desktop/Python/Scripts$ which python
/usr/bin/python
USER:~/Desktop/Python/Scripts$ which python3
/home/linuxbrew/.linuxbrew/bin/python3
Next, upon executing a python-script in VS Code via Ctrl + Alt + P the following error (along with the entire output) occurs:
USER:~/Desktop/Python/Scripts$
/home/linuxbrew/.linuxbrew/bin/python3 -c "from IPython import start_ipython; start_ipython()" /home/cripts/Master/Import_export/AERMET/import_and_manip_sitespec_meteodata_AERMET.py
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/numpy/core/__init__.py in <module>
21 try:
---> 22 from . import multiarray
23 except ImportError as exc:
/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/numpy/core/multiarray.py in <module>
11
---> 12 from . import overrides
13 from . import _multiarray_umath
/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/numpy/core/overrides.py in <module>
6
----> 7 from numpy.core._multiarray_umath import (
8 add_docstring, implement_array_function, _get_implementing_args)
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
~/Dokumente/Allgemeines_material/Sonstiges/Programming/Python/Scripts/Master/Import_export/AERMET/import_and_manip_sitespec_meteodata_AERMET.py in <module>
40
41 import re
---> 42 import numpy as np
43 import pandas as pd
44 from tqdm import tqdm
/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/numpy/__init__.py in <module>
138 from . import _distributor_init
139
--> 140 from . import core
141 from .core import *
142 from . import compat
/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/numpy/core/__init__.py in <module>
46 """ % (sys.version_info[0], sys.version_info[1], sys.executable,
47 __version__, exc)
---> 48 raise ImportError(msg)
49 finally:
50 for envkey in env_added:
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.8 from "/home/linuxbrew/.linuxbrew/opt/python@3.8/bin/python3.8"
* The NumPy version is: "1.19.0"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Nevertheless, numpy is already up-to-date:
USER:~/Desktop/Python/Scripts$ pip install numpy
Requirement already satisfied: numpy in /home/linuxbrew/.linuxbrew/lib/python3.8/site-packages (1.19.0)
Also, in the settings.json file of VS Code, I declared for playing safe the following:
"python.autoComplete.extraPaths": [
"/home/andylu/Dokumente/Allgemeines_material/Sonstiges/Programming/Python/Scripts",
"/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages",
"/home/linuxbrew/.linuxbrew/Cellar/python@3.8/3.8.3_2/lib/python3.8/site-packages"
]
In my .bashrc - file, I already updated everything with no avail:
# Link to own and official python scripts and packages
export PYTHONPATH=/home/andylu/Dokumente/Allgemeines_material/Sonstiges/Programming/Python/Scripts:/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages:/home/linuxbrew/.linuxbrew/Cellar/python@3.8/3.8.3_2/lib/python3.8/site-packages
# Homebrew
export PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"
All of these issues caught me totally off guard and I feel utterly lost. I wanted to continue on my project today, but spend already 2 hours on the internet finding out that none of the proposed solutions worked for me, wherefore I started this issue here.
The solution to the above-mentioned problems was that a workaround in which every module/package causing problems needed to be uninstalled first and then reinstalled from scratch.
This is apparently different than simply trying to update an already incorrectly installed package.
I presume that this was caused through moving the python site-packages installed in the /../python3.7/site-packages to the new /../python3.8/site-packages directory manually.
Apparently, not all packages will work well afterwards and need to be uninstalled and reinstalled instead of just updated.
In more detail, I did pip uninstall packagename followed by pip install packagename with every package causing problems, which were fortunately less than 10.
Instead of using pip install ... one can also employ python3 -m pip install ... if that works better or if the pip-binary is somehow corrupted as it was initially my case (see question above).
After having resolved the issues via the described workaround, naturally a follow-up question emerged as to how you can migrate the site-packages more elegantly to a new python 3.x version.