Long story short, I understand what my problem comes from, but haven't been able to find a solution.
The error I get is pasted below:
pip install scipy (and other variations like --upgrade, making sure pip is most recent, etc. - almost certain its not a pip issue)
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [13 lines of output]
+ meson setup --prefix=C:\users\w\appdata\local\programs\python\python38-32 C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c\.mesonpy-wjzxo0cz\build --native-file=C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c\.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
The Meson build system
Version: 1.2.1
Source dir: C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c
Build dir: C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c\.mesonpy-wjzxo0cz\build
Build type: native build
Project name: SciPy
Project version: 1.10.1
Activating VS 16.3.9
..\..\meson.build:1:0: ERROR: Compiler cl cannot compile programs.
A full log can be found at C:\Users\w\AppData\Local\Temp\pip-install-bf55dgso\scipy_82bfcabbf1854ff981dec7247a79719c\.mesonpy-wjzxo0cz\build\meson-logs\meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
From what I can gather, SciPy has issues with Windows and this is a well documented issue for the past decade or so - something to do with the lack of a compiler for - and issue that Linux/MacOS doesnt have because of gcc. Here is my problem: most solutions reference http://www.lfd.uci.edu/~gohlke/pythonlibs/ - telling me to download the matching .whl file and it should be fixed, as in the answer here. Issue is, the gohlke link is broken in 2023 and directs to a Page Not Found.
While not a hard requirement for the fix, I personally would rather not download files of unknown origin from the internet - however if someone has a copy of the .whl for Windows 10 64-bit, I guess it's better than wiping my laptop and making it run Ubuntu.
You have (most likely unintentionally) installed a 32 bit version of python 3.8, apparent from your error message, which indicates the path of your python installation
C:\users\w\appdata\local\programs\python\python38-32
Unintstall it completely and install a 64 bit version of python. Then you should not face any more issues.
Since there are no whl files for 32 bit windows released in the current scipy (not even sure if there are any for other versions), pip defaults to building from source. Something that can be quite tricky for libraries with C backends