I'm currently trying to install SciPy from source, but I keep running into problems. I'm following the instructions on https://scipy.github.io/devdocs/building/linux.html.
What I did:
sudo apt-get install gcc g++ gfortran python3-dev libopenblas-dev liblapack-dev.What does not work:
Both python3 setup.py build and python3 setup.py install don't work, with or without sudo. Moreover, trying to install it with pip, by typing pip3 install . also does not work.
The problem seems to be related to the fact that a file called ./scipy/_lib/highs/CMakeLists.txt can't be found. Is this file something supposed to be generated automatically?
Console output of pip3 install .:
Processing /mnt/data/Code/Python scripts/scipy
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [178 lines of output]
scipy/linalg/_generate_pyx.py: all files up-to-date
scipy/special/_generate_pyx.py: all files up-to-date
Running scipy/linalg/_generate_pyx.py
Running scipy/special/_generate_pyx.py
Running scipy/stats/_generate_pyx.py
scipy/cluster/_hierarchy.pyx has not changed
scipy/cluster/_optimal_leaf_ordering.pyx has not changed
scipy/linalg/_solve_toeplitz.pyx has not changed
scipy/linalg/_cythonized_array_utils.pyx has not changed
scipy/linalg/_decomp_update.pyx.in has not changed
scipy/ndimage/src/_cytest.pyx has not changed
scipy/ndimage/src/_ni_label.pyx has not changed
scipy/optimize/_bglu_dense.pyx has not changed
scipy/optimize/_group_columns.pyx has not changed
scipy/optimize/cython_optimize/_zeros.pyx.in has not changed
scipy/optimize/_lsq/givens_elimination.pyx has not changed
scipy/optimize/tnc/_moduleTNC.pyx has not changed
scipy/optimize/_highs/cython/src/_highs_constants.pyx has not changed
scipy/optimize/_highs/cython/src/_highs_wrapper.pyx has not changed
scipy/optimize/_trlib/_trlib.pyx has not changed
scipy/signal/_max_len_seq_inner.pyx has not changed
scipy/signal/_peak_finding_utils.pyx has not changed
scipy/signal/_sosfilt.pyx has not changed
scipy/signal/_spectral.pyx has not changed
scipy/signal/_upfirdn_apply.pyx has not changed
scipy/sparse/_csparsetools.pyx.in has not changed
scipy/sparse/csgraph/_flow.pyx has not changed
scipy/sparse/csgraph/_matching.pyx has not changed
scipy/sparse/csgraph/_min_spanning_tree.pyx has not changed
scipy/sparse/csgraph/_reordering.pyx has not changed
scipy/fftpack/convolve.pyx has not changed
scipy/sparse/csgraph/_tools.pyx has not changed
scipy/interpolate/_bspl.pyx has not changed
scipy/spatial/_ckdtree.pyx has not changed
scipy/io/matlab/_mio5_utils.pyx has not changed
scipy/spatial/_qhull.pyx has not changed
scipy/spatial/_voronoi.pyx has not changed
scipy/spatial/transform/_rotation.pyx has not changed
scipy/special/cython_special.pyx has not changed
scipy/special/_ellip_harm_2.pyx has not changed
scipy/special/_test_round.pyx has not changed
scipy/special/_ufuncs.pyx has not changed
scipy/interpolate/interpnd.pyx has not changed
scipy/sparse/csgraph/_traversal.pyx has not changed
scipy/stats/_qmc_cy.pyx has not changed
scipy/stats/_sobol.pyx has not changed
scipy/stats/_stats.pyx has not changed
scipy/io/matlab/_streams.pyx has not changed
scipy/linalg/cython_blas.pyx has not changed
scipy/stats/_unuran/unuran_wrapper.pyx has not changed
scipy/stats/_boost/src/beta_ufunc.pyx has not changed
scipy/stats/_boost/src/binom_ufunc.pyx has not changed
scipy/stats/_boost/src/hypergeom_ufunc.pyx has not changed
scipy/special/_ufuncs_cxx.pyx has not changed
scipy/_lib/messagestream.pyx has not changed
scipy/_lib/_ccallback_c.pyx has not changed
scipy/spatial/_hausdorff.pyx has not changed
scipy/_lib/_test_deprecation_def.pyx has not changed
scipy/stats/_biasedurn.pyx has not changed
scipy/stats/_levy_stable/levyst.pyx has not changed
scipy/linalg/cython_lapack.pyx has not changed
scipy/linalg/_matfuncs_sqrtm_triu.pyx has not changed
scipy/cluster/_vq.pyx has not changed
scipy/sparse/csgraph/_shortest_path.pyx has not changed
scipy/stats/_boost/src/nbinom_ufunc.pyx has not changed
scipy/special/_comb.pyx has not changed
scipy/interpolate/_ppoly.pyx has not changed
scipy/_lib/_test_deprecation_call.pyx has not changed
scipy/io/matlab/_mio_utils.pyx has not changed
Cythonizing sources
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_lapack_info:
C compiler: x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC
creating /tmp/tmpxeu9x3zd/tmp
creating /tmp/tmpxeu9x3zd/tmp/tmpxeu9x3zd
compile options: '-c'
x86_64-linux-gnu-gcc: /tmp/tmpxeu9x3zd/source.c
x86_64-linux-gnu-gcc -pthread /tmp/tmpxeu9x3zd/tmp/tmpxeu9x3zd/source.o -L/usr/lib/x86_64-linux-gnu -lopenblas -o /tmp/tmpxeu9x3zd/a.out
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/x86_64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/x86_64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
blas_mkl_info:
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
blis_info:
libraries blis not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_info:
C compiler: x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC
creating /tmp/tmph3qne29w/tmp
creating /tmp/tmph3qne29w/tmp/tmph3qne29w
compile options: '-c'
x86_64-linux-gnu-gcc: /tmp/tmph3qne29w/source.c
x86_64-linux-gnu-gcc -pthread /tmp/tmph3qne29w/tmp/tmph3qne29w/source.o -L/usr/lib/x86_64-linux-gnu -lopenblas -o /tmp/tmph3qne29w/a.out
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/x86_64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/x86_64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
setup.py:390: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-qr6r8y4e'), proceeding with generating Cython sources and expanding templates
warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
Running from SciPy source directory.
/usr/local/lib/python3.8/dist-packages/numpy/distutils/system_info.py:933: UserWarning: Specified path /usr/local/include/python3.8 is invalid.
return self.get_paths(self.section, key)
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
self.run_setup()
File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 281, in run_setup
super(_BuildMetaLegacyBackend,
File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 174, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 532, in <module>
setup_package()
File "setup.py", line 528, in setup_package
setup(**metadata)
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/core.py", line 135, in setup
config = configuration()
File "setup.py", line 438, in configuration
config.add_subpackage('scipy')
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 1014, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 980, in get_subpackage
config = self._get_configuration_from_setup_py(
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 922, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "/mnt/data/Code/Python scripts/scipy/scipy/setup.py", line 18, in configuration
config.add_subpackage('optimize')
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 1014, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 980, in get_subpackage
config = self._get_configuration_from_setup_py(
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 922, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "/mnt/data/Code/Python scripts/scipy/scipy/optimize/setup.py", line 130, in configuration
config.add_subpackage('_highs')
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 1014, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 980, in get_subpackage
config = self._get_configuration_from_setup_py(
File "/usr/local/lib/python3.8/dist-packages/numpy/distutils/misc_util.py", line 922, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "/mnt/data/Code/Python scripts/scipy/scipy/optimize/_highs/setup.py", line 63, in configuration
_major_dot_minor = _get_version(
File "/mnt/data/Code/Python scripts/scipy/scipy/optimize/_highs/setup.py", line 50, in _get_version
with open(CMakeLists, 'r', encoding='utf-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/data/Code/Python scripts/scipy/scipy/_lib/highs/CMakeLists.txt'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
As you can see on git the folder highs is included there, but as a submodule, which you have probably not downloaded when you cloned the repo.
Go to the repo main dir and run
git submodule init
git submodule update
Then confirm that the specified folder exists and is populated