Minimizing SSE using Scipy.optimize minimize

Question

I am trying to optimize SSE (sum of squared error) of a function using scipy.optimize. To test with, I created a simple problem as below code. But the optimized parameters output by scipy never makes SSE=0. Can someone help me to understand, where am I going wrong.

I tried to cross check with the SSE calculated by my code with the one computed in excel. It matched. Then I used minimize function to minimize that SSE function, the ones computed by Scipy is not matching with the hand calculated ones. The function I used to is of form (y=ax+b). Below is the code

import numpy as np
from scipy.optimize import minimize


e=np.array([0,2])
sig1=np.array([0,200])
k = [10,10]
#n = 0.2
coe=np.array([k[0],k[1]])

def sig2(e):
    v=(k[0]*e)+ k[1]
    SEzip = zip(sig1, v)
    sse = 0
    for y in SEzip:
        sse += np.power((y[0] - y[1]),2)
    return sse

print (sig2(e))
def f(coe):
   print(coe)
   return f
result = minimize(sig2,coe,method='Nelder-Mead',callback=(f),options={'xtol': 1e-6,'ftol':1e-06,'maxiter':50000,'disp': True,'adaptive' : True})

print(result)

Answer 1

You were printing your x0 aka coe in here, I edited your code and shortened your objective function sig2() into one line then edited your callback to display the variable tested and its equivalent objective function value. Now you can clearly see that sse=0 is reached.

import numpy as np
from scipy.optimize import minimize

# for prettier numpy prints
np.set_printoptions(precision = 6)

# init 
e    = np.array([0,2])
sig1 = np.array([0,200])
k    = [10, 10]
coe  = np.array([k[0], k[1]])

# define objective function
def sig2(e):
    return sum([np.power((y[0] - y[1]), 2) for y in zip(sig1, (k[0]*e)+ k[1])])

# define callback
def f(e):
   print("e: %25s | sig2(e): %5s" % (e,round(sig2(e), 6)))

# optimize
result = minimize(sig2,
                  coe,
                  method   = 'Nelder-Mead',
                  callback = f,
                  options  = {'xtol': 1e-6,'ftol':1e-06,
                              'maxiter':50000,'disp': True,'adaptive' : True})

print(result)

Output:

...
e:     [-1.000053 18.999751] | sig2(e): 6e-06
e:     [-1.000062 19.000109] | sig2(e): 2e-06
e:     [-1.000062 19.000109] | sig2(e): 2e-06
e:     [-1.000062 19.000109] | sig2(e): 2e-06
e:     [-0.999934 18.999981] | sig2(e):   0.0
e:     [-1.000049 18.999979] | sig2(e):   0.0
e:     [-1.000027 19.000044] | sig2(e):   0.0
e:     [-0.999986 18.999996] | sig2(e):   0.0
e:     [-0.999986 18.999996] | sig2(e):   0.0
e:     [-0.999986 18.999996] | sig2(e):   0.0
e:     [-1.000009 18.999993] | sig2(e):   0.0
e:     [-1.000009 18.999993] | sig2(e):   0.0
e:     [-0.999995 19.      ] | sig2(e):   0.0
e:     [-0.999995 19.      ] | sig2(e):   0.0
e:     [-1.000003 18.999998] | sig2(e):   0.0
e:     [-1.       19.000002] | sig2(e):   0.0
e:     [-0.999998 19.      ] | sig2(e):   0.0
e:     [-1.000001 18.999999] | sig2(e):   0.0
e:     [-1.       19.000001] | sig2(e):   0.0
e:     [-0.999999 19.      ] | sig2(e):   0.0
e:                 [-1. 19.] | sig2(e):   0.0
e:                 [-1. 19.] | sig2(e):   0.0
e:                 [-1. 19.] | sig2(e):   0.0
Optimization terminated successfully.
         Current function value: 0.000000
         Iterations: 56
         Function evaluations: 110
 final_simplex: (array([[-1., 19.],
       [-1., 19.],
       [-1., 19.]]), array([6.221143e-12, 1.914559e-11, 1.946860e-11]))
           fun: 6.2211434216849394e-12
       message: 'Optimization terminated successfully.'
          nfev: 110
           nit: 56
        status: 0
       success: True
             x: array([-1., 19.])