Search code examples
pythonnumpyscipycurve-fitting

Why does using scipy.optimize.curvefit with numpy.piecewise throw Attribute and Runtime Errors?


Here is a reduced example of some curve fitting code I am working on.

The piecewise function is just composed of two constant functions and two linear functions. As you can see, the function behaves fine when assigning y_space but scipy.optimize's curve_fit eventually evokes an error from numpy. Unfortunately, I don't know enough about numpy's inner workings to begin to diagnose this. I haven't seen anyone else report this kind of error. (I am encountering a similar but different error when writing out the function without using np.piecewise, but I will stay focused on this issue.) As usual, any help is appreciated.

Code:

# Imports
from scipy.optimize import curve_fit
import numpy as np
import matplotlib.pyplot as plt

plt.style.use("dark_background")

# Piecewise function
def my_piecewise(
    x: float,
    a: float,
    b: float,
    c: float,
    d: float,
    precision: float = 64,
):
    domain_length = 512
    if precision == 32:
        domain_length = 1024

    if precision not in (32, 64):
        raise ValueError("Precision must be either 64 or 32")

    y = np.piecewise(
        x,
        [
            x <= 0,
            x < domain_length,
            (x >= domain_length) & (x < 2 * domain_length),
            x >= 2 * domain_length,
        ],
        [
            lambda x: 0,
            lambda x: a + ((b - a) / (domain_length - 8)) * x,
            lambda x: c + ((d - c) / domain_length) * (x - domain_length),
            lambda x: d,
        ],
    )

    return y


# p0 parameters for curve_fit
init_guess = [
    0.006,
    0.0065,
    0.006,
    0.0065,
]

# x and y data
x_space = np.linspace(8, 4096, 512, endpoint=True)
y_space = my_piecewise(x_space, *init_guess)

# plotting with matplotlib works fine
plt.plot(x_space, y_space)
for i in [512, 1024, 2048]:
    plt.vlines(i, 0.004, 0.008, color="white", linewidth=0.75)
plt.show()

# curve_fit
param, param_cov = curve_fit(
    f=my_piecewise,
    xdata=x_space,
    ydata=y_space,
    p0=init_guess,
)

Matplotlib plot:

Error:

Traceback (most recent call last):
  File "/path/to/conda/env/lib/python3.12/site-packages/numpy/_core/arrayprint.py", line 34, in <module>
    from . import numerictypes as _nt
  File "/path/to/conda/env/lib/python3.12/site-packages/numpy/_core/numerictypes.py", line 102, in <module>
    from ._type_aliases import (
  File "/path/to/conda/env/lib/python3.12/site-packages/numpy/_core/_type_aliases.py", line 38, in <module>
    allTypes[_abstract_type_name] = getattr(ma, _abstract_type_name)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'numpy.core.multiarray' has no attribute 'signedinteger'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/garrett/Documents/projects/benchmark-data-fitting/temp.py", line 63, in <module>
    param, param_cov = curve_fit(
                       ^^^^^^^^^^
  File "/path/to/conda/env/lib/python3.12/site-packages/scipy/optimize/_minpack_py.py", line 1007, in curve_fit
    res = leastsq(func, p0, Dfun=jac, full_output=1, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/conda/env/lib/python3.12/site-packages/scipy/optimize/_minpack_py.py", line 439, in leastsq
    retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/conda/env/lib/python3.12/site-packages/scipy/optimize/_minpack_py.py", line 519, in _memoized_func
    if np.all(_memoized_func.last_params == params):
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/conda/env/lib/python3.12/site-packages/numpy/core/_internal.py", line 855, in array_ufunc_errmsg_formatter
    args_string = ', '.join(['{!r}'.format(arg) for arg in inputs] +
                             ^^^^^^^^^^^^^^^^^^
RuntimeError: Unable to configure default ndarray.__repr__

Solution

  • There might be something wrong with your specific python installation.

    I could not reproduce your bug either with virtual environment (on Windows with Anaconda) or using Docker (on Linux).

    The following environment:

    python -m venv bug
    bug\Scripts\activate.bat
    python -m pip install numpy==2.0.1 scipy==1.14.0 matplotlib==3.8.4
    python bug.py
    

    And the following image:

    FROM python:3.12-bullseye   
    RUN python -m pip install numpy==2.0.1 scipy==1.14.0 matplotlib==3.8.4
    COPY bug.py .
    CMD python bug.py
    

    Run fine and return without any error.

    Solving your issue will necessarily require to find what is different in your setup. Did you tweak some files ? Have you tried to reinstall all packages ? Do you have errors when you use the conda installation manager ?