Scipy minimize is giving me a "desired error not necessarily achieved due to precision loss", and my code seemingly is correct

Question

Essentially what the title says. Getting some initial practice with scipy minimize and can't figure out why it won't converge.

I have my prediction model as follows:

def predict(X,betas):
    y_hat = np.dot(X,betas)
    return y_hat
X = np.array([[1,0],[1,-1],[1,2]])
betas = np.array([0.1,0.3])
y_hat = predict(X,betas)
print(y_hat)

which works as expected.

Then, I have my loss/gradient function as follows:

def lossRSS(betas,X,y):
    y_hat = predict(X, betas)
    res = y_hat-y
    rss = np.sum(res * res)
    gradient = -2 * np.transpose(X).dot(res)
    return (rss, gradient)
X = np.array([[1,0],[1,-1],[1,2]])
betas = np.array([0.1,0.3])
y = np.array([0,0.4,2])
lossRSS(betas,X,y)

which also works as expected.

Finally, I get to my minimization function which I have implemented as follows:

def minimization(X, y, lossfuncn):
    betas = np.array([0.1,0.3])
    result = so.minimize(lossfuncn, betas, args=(X, y), jac=True)
    print(result)

X = np.array([[1,0],[1,-1],[1,2]])
y = np.array([0,0.4,2]) 
minimization(X,y,lossRSS)

But I get the following output:

fun: 2.06
 hess_inv: array([[1, 0],
       [0, 1]])
      jac: array([3.6, 4. ])
  message: 'Desired error not necessarily achieved due to precision loss.'
     nfev: 53
      nit: 0
     njev: 41
   status: 2
  success: False
        x: array([0.1, 0.3])

and I cannot figure out why. Is there a parameter I'm misusing in the optimize function? I'm not too sharp on the theory behind the minimization methods, but from my knowledge of how minimize and optimize operate, it should work.

Any insights would be very appreciated!

Answer 1

My problem was that I had

res=y_hat-y

instead of

res=y-y_hat

An elementary mistake, which is probably why I overlooked it. Decided to answer this instead of deleting to remind people that maybe the mistake is something super silly that they think they're above!