How do I fix this error IndexError: too many indices for array

I am trying to fit a piece wise function to a time series that I have, where the first part is an exponential decay and the second part is linear, I have looked up many examples, but I keep getting that error. This is a python program.

Here is my code

def piece_wise(t, t0, a, b, c, d):

return np.piecewise(t, [t <= t0, t > t0], [lambda t:a*np.exp(b*-t) + d, lambda t:c*t + d])

t = filtered["Time (s)"]

y = filtered["Average"]

p, e = optimize.curve_fit(piece_wise, t, y)

The filtered data frame has 2187 rows

This is the full error

  File "/opt/anaconda3/lib/python3.8/site-packages/numpy/lib/function_base.py", line 626, in piecewise
    vals = x[condlist[k]]

IndexError: too many indices for array

Now looking at the program function_base.py, k is a loop variable looped over the number of elements in the condition list passed to np.piecewise. What am I doing wrong here?

Thanks!

Solution

First of all, before doing something complicated like passing your function to scipy.optimize, you want to check that it works.

You can simply call it with some arbitrary data and parameters:

t = filtered["Time (s)"]
t0 = 5; a = 0; b = 0; c = 0; d = 0;
piece_wise(t, t0, a, b, c, d)

This produces the traceback:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-24-e4aa1198c545> in <module>
----> 1 piece_wise(t, t0, a, b, c, d)

<ipython-input-16-d251be3d390b> in piece_wise(t, t0, a, b, c, d)
      1 def piece_wise(t, t0, a, b, c, d):
----> 2     return np.piecewise(t, [t <= t0, t > t0], [lambda t:a*np.exp(b*-t) + d, lambda t:c*t + d])
      3 

<__array_function__ internals> in piecewise(*args, **kwargs)

~/anaconda3/envs/torch/lib/python3.9/site-packages/numpy/lib/function_base.py in piecewise(x, condlist, funclist, *args, **kw)
    612             y[cond] = func
    613         else:
--> 614             vals = x[cond]
    615             if vals.size > 0:
    616                 y[cond] = func(vals, *args, **kw)

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

This confirms that the problem is nothing to do with scipy.optimize.

Also, it tells us that the error occurred on this line:

return np.piecewise(t, [t <= t0, t > t0], [lambda t:a*np.exp(b*-t) + d, lambda t:c*t + d])

Let's break it down...

f1, f2 = lambda t:a*np.exp(b*-t) + d, lambda t:c*t + d
f1(0)
f2(0)

No errors, so these two functions are ok...

Let's look at the documentation for np.piecewise

piecewise(x, condlist, funclist, *args, **kw)
    Evaluate a piecewise-defined function.
    
    Given a set of conditions and corresponding functions, evaluate each
    function on the input data wherever its condition is true.
    
    Parameters
    ----------
    x : ndarray or scalar
        The input domain.
    condlist : list of bool arrays or bool scalars
        ....
    funclist : list of callables, f(x,*args,**kw), or scalars
        ....

Ah look! It says x should be an ndarray or scalar.

We are passing a Pandas Series.

Try this:

t = filtered["Time (s)"].to_numpy()
t0 = 5; a = 0; b = 0; c = 0; d = 0;
piece_wise(t, t0, a, b, c, d)

Out: array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])

Now it seems to work. I think this might be your problem.

Hopefully I have helped you with a means to debug such problems in future. Always breakdown problems into smaller pieces and test each piece separately.