scipy minimize on function with multiple inputs, and optimize with multiple bounds

Question

I am trying to optimize a function with 2 inputs, each being a list of numbers. I created these similar but simpler version of the function:

w1 = [1,2,3] 
w2 = [4,5,6] 
w = [w1,w2]

def objective(x):
    a = x[0][0]**2+x[0][1]**2+x[0][2]**2+x[1][0]**2+x[1][1]**2+x[1][2]**2
    return a 
bnds_1 = tuple((0.1, 1) for w in w1) 
bnds_2 = tuple((0,0.5) for w in w2)

result = minimize(objective,x0=w,bounds=(bnds_1,bnds_2)) 
result

where the bound for each number in w1 is (0.1,1) and bound for each number in w2 is (0,0.5)

I get the following error when running the code:

ValueError: length of x0 != length of bounds

Could you please advise on what's wrong with this?

P.S. I know that I could put both w1 and w2 in 1 list and just call the different items, but was just wondering why this method with 2 inputs doesn't work

Answer 1

Looks like I need to quote the relevant parts of the minimize docs

x0 - ndarray, shape (n,)
Initial guess. Array of real elements of size (n,), 
where n is the number of independent variables.

np.array([w1,w2]) is a (2,3) array. Check the minimize code, but I think it will be flattened to (6,). You could also test the x that is passed to objective.

In [96]: w1 = [1,2,3] 
    ...: w2 = [4,5,6] 
    ...: w = [w1,w2]

In [97]: def objective(x):
    ...:     print('x.shape', x.shape)
    ...:     a = x[0][0]**2+x[0][1]**2+x[0][2]**2+x[1][0]**2+x[1][1]**2+x[1][2]**2
    ...:     return a

Run with the 2d initial condition, the objective returns:

In [99]: objective(np.array(w))
x.shape (2, 3)
Out[99]: 91

but called via minimize:

In [100]: minimize(objective, w)
x.shape (6,)
3     a = x[0][0]**2+x[0][1]**2+x[0][2]**2+x[1][0]**2+x[1][1]**2+x[1][2]**2
IndexError: invalid index to scalar variable.

x is (6,) but your function expects a 2 element list of 3 element lists, or a (2,3) array.

And that's not even getting to the bounds.

So with n being 6, it is expecting the bounds to be 6 tuples:

In [102]: minimize(objective,x0=w,bounds=(bnds_1,bnds_2))
    282     bounds = [(None, None)] * n
    283 if len(bounds) != n:
--> 284     raise ValueError('length of x0 != length of bounds')

The key when using scipy functions like minimize is that YOU have to conform to its usage. It is in control, not you.

Adapting objective to work with this (6,) input:

In [110]: def objective(x):
     ...:     #print('x.shape', x.shape)
     ...:     x = x.reshape(2,3)
     ...:     a = x[0][0]**2+x[0][1]**2+x[0][2]**2+x[1][0]**2+x[1][1]**2+x[1][2]**2
     ...:     return a
     ...:     

And changing the bounds to by (6,2):

In [111]: minimize(objective,x0=w,bounds=np.array((bnds_1,bnds_2)).reshape(-1,2))
Out[111]: 
      fun: 0.030000000000000006
 hess_inv: <6x6 LbfgsInvHessProduct with dtype=float64>
      jac: array([2.00000010e-01, 2.00000010e-01, 2.00000010e-01, 1.00613962e-08,
       1.00613962e-08, 1.00613962e-08])
  message: 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
     nfev: 14
      nit: 1
     njev: 2
   status: 0
  success: True
        x: array([0.1, 0.1, 0.1, 0. , 0. , 0. ])