After installation of scipy 1.14.1 a previously viable Python program now fails.
The original program is more complex so here's a MRE:
import pandas as pd
import plotly.express as px
if __name__ == "__main__":
data = {
"Date": [0, 7, 14, 21, 28],
"Value": [100, 110, 120, 115, 122]
}
df = pd.DataFrame(data)
px.scatter(df, x="Date", y="Value", trendline="ols")
Here's the stack trace (username obfuscated):
Traceback (most recent call last):
File "/Users/****/Python/Aug22.py", line 12, in <module>
tl = px.scatter(
^^^^^^^^^^^
File "/Users/****/venv/lib/python3.12/site-packages/plotly/express/_chart_types.py", line 66, in scatter
return make_figure(args=locals(), constructor=go.Scatter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/****/venv/lib/python3.12/site-packages/plotly/express/_core.py", line 2267, in make_figure
patch, fit_results = make_trace_kwargs(
^^^^^^^^^^^^^^^^^^
File "/Users/****/venv/lib/python3.12/site-packages/plotly/express/_core.py", line 361, in make_trace_kwargs
y_out, hover_header, fit_results = trendline_function(
^^^^^^^^^^^^^^^^^^^
File "/Users/****/venv/lib/python3.12/site-packages/plotly/express/trendline_functions/__init__.py", line 43, in ols
import statsmodels.api as sm
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/api.py", line 136, in <module>
from .regression.recursive_ls import RecursiveLS
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/regression/recursive_ls.py", line 14, in <module>
from statsmodels.tsa.statespace.mlemodel import (
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/mlemodel.py", line 33, in <module>
from .simulation_smoother import SimulationSmoother
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/simulation_smoother.py", line 11, in <module>
from .kalman_smoother import KalmanSmoother
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/kalman_smoother.py", line 11, in <module>
from statsmodels.tsa.statespace.representation import OptionWrapper
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/representation.py", line 10, in <module>
from .tools import (
File "/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/tools.py", line 14, in <module>
from . import (_initialization, _representation, _kalman_filter,
File "statsmodels/tsa/statespace/_initialization.pyx", line 1, in init statsmodels.tsa.statespace._initialization
ImportError: dlopen(/Users/****/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/_representation.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_npy_cabs'
If I revert to scipy 1.14.0 this error does not occur.
Is there anything I can do that would allow me to run with scipy 1.14.1?
Platform:
macOS 14.6.1
python 3.12.5
Apple M2
Update: This issue has now been fixed in statsmodels 0.14.3.
The rest of this answer is of historical interest only.
This looks like a bug to me, so I filed issues against SciPy, statsmodels, and the statsmodels maintainer filed an issue against NumPy.
Here's a summary of what I've learned from those issues.
In NumPy 2.1.0, NumPy started marking certain functions with __attribute__((visibility("hidden")), with the effect that extensions built against NumPy would not re-export symbols exported by NumPy. The pull request doing this is here.
These are the cabs related symbols exported by NumPy prior to 2.1.0:
$ nm -gU venv/lib/python3.12/site-packages/numpy/linalg/_umath_linalg.cpython-312-darwin.so | grep cabs 1 ↵
00000000000148c0 T _npy_cabs
000000000001487c T _npy_cabsf
0000000000014904 T _npy_cabsl
And after:
$ nm -gU venv/lib/python3.12/site-packages/numpy/linalg/_umath_linalg.cpython-312-darwin.so | grep cabs
It stopped exporting _npy_cabs.
In SciPy 1.14.1, SciPy switched from being built against NumPy 2.0.1 to being built against 2.1.0.
As a consequence, it also stopped exporting the _npy_cabs symbol.
These are the cabs related symbols being exported prior to 1.14.1:
$ nm -gU venv/lib/python3.12/site-packages/scipy/special/_ufuncs.cpython-312-darwin.so | grep cabs
000000000007c248 T _npy_cabs
000000000007c204 T _npy_cabsf
000000000007c28c T _npy_cabsl
And after:
$ nm -gU venv/lib/python3.12/site-packages/scipy/special/_ufuncs.cpython-312-darwin.so | grep cabs
It also stopped exporting _npy_cabs. (The npy stands for NumPy, so it is a little strange for SciPy to provide this function.)
In summary, you have three choices of ways to solve the problem.
Downgrade SciPy from 1.14.1 to 1.14.0.
As a consequence, you would be missing these fixes.
Downgrade NumPy from 2.1.0 to 2.0.1.
As a consequence, you would be missing these fixes and new features.
Wait for a fix from statsmodels for the issue. They would need to adjust the way they are linking NumPy.
This has now been fixed in statsmodels 0.14.3.
Issue links: scipy statsmodels numpy