python 5 (dim 1) != 1 (dim 0)

I am trying to use scipy.optimize to solve a quadratic programming.

def objFun(vector):
    "input value is a vector parameter"
    return ((vector.transpose()*Q*vector + b.transpose()*vector)[0] + c).item()

def scipy_result():
    init = np.matrix([[1] for i in range(5)])
    res = optimize.minimize(objFun, init, method="CG")
    print("value of x^* is", res.x, '\n')

if __name__ == "__main__":
    scipy_result()

I don't there is any error in the dimension of multiplication and stuff.

But

Traceback (most recent call last):

  File "C:\Users\Siyuan Xu\OneDrive - purdue.edu\Purdue Courses\CS 520\programming hw 1\homework 1.py", line 91, in <module>
    scipy_result()

  File "C:\Users\Siyuan Xu\OneDrive - purdue.edu\Purdue Courses\CS 520\programming hw 1\homework 1.py", line 63, in scipy_result
    res = optimize.minimize(objFun, init, method="CG")

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\_minimize.py", line 610, in minimize
    return _minimize_cg(fun, x0, args, jac, callback, **options)

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\optimize.py", line 1423, in _minimize_cg
    sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\optimize.py", line 261, in _prepare_scalar_function
    sf = ScalarFunction(fun, x0, args, grad, hess,

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 76, in __init__
    self._update_fun()

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 166, in _update_fun
    self._update_fun_impl()

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 73, in update_fun
    self.f = fun_wrapped(self.x)

  File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 70, in fun_wrapped
    return fun(x, *args)

  File "C:\Users\Siyuan Xu\OneDrive - purdue.edu\Purdue Courses\CS 520\programming hw 1\homework 1.py", line 30, in objFun
    return ((vector.transpose()*Q*vector + b.transpose()*vector)[0] + c).item()

  File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\matrixlib\defmatrix.py", line 220, in __mul__
    return N.dot(self, asmatrix(other))

  File "<__array_function__ internals>", line 5, in dot

ValueError: shapes (1,5) and (1,5) not aligned: 5 (dim 1) != 1 (dim 0)

I am confused. And I tried. Making input an array (change the obj function as well) and it also does not work. Please help me to see how to fix this. Thanks a lot!

Solution

In your problem statement, Q,b,c are not defined or described. But the error makes me suspect that Q is a np.matrix.

Let's explore a slightly simpler function:

In [58]: from scipy import optimize
In [59]: def f(x):
    ...:     print(x, x.shape)
    ...:     Q = np.matrix([[1,2],[3,4]])
    ...:     res = x.T*Q*x
    ...:     print(res, res.shape)
    ...:     return res
    ...:

It's often a good idea to test the objective function by itself. np.minimize says the initial variable should be 1d, (n,) shape, but let's see how various arguments work:

1d (2,) shape:

In [60]: f(np.arange(2))
[0 1] (2,)
Traceback (most recent call last):
  File "<ipython-input-60-ef04d08d3a6e>", line 1, in <module>
    f(np.arange(2))
  File "<ipython-input-59-ef8f7ef56c80>", line 4, in f
    res = x.T*Q*x
  File "/usr/local/lib/python3.8/dist-packages/numpy/matrixlib/defmatrix.py", line 218, in __mul__
    return N.dot(self, asmatrix(other))
  File "<__array_function__ internals>", line 5, in dot
ValueError: shapes (1,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0)

In detail:

In [65]: Q=np.matrix([[1,2],[3,4]])
In [66]: x=np.arange(2)
In [67]: x*Q                    # first matrix product (because of Q)
Out[67]: matrix([[3, 4]])       # (1,2) shape
In [68]: (x*Q)*x                # second product
Traceback (most recent call last):
  File "<ipython-input-68-94f5369405fc>", line 1, in <module>
    (x*Q)*x
  File "/usr/local/lib/python3.8/dist-packages/numpy/matrixlib/defmatrix.py", line 218, in __mul__
    return N.dot(self, asmatrix(other))
  File "<__array_function__ internals>", line 5, in dot
ValueError: shapes (1,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0)

When used with Q, np.matrix(np.arange(2)) produces a (1,2) matrix([[0, 1]]) matrix.

But if x starts as (2,1) shape then the double product does work:

In [72]: x=np.arange(2)[:,None]
In [73]: (x.T*Q)*x
Out[73]: matrix([[4]])
In [74]: f(x)
[[0]
 [1]] (2, 1)
[[4]] (1, 1)
Out[74]: matrix([[4]])

But if I plus that f and x into minimize I get an error, because the (2,1) array has been 'reduced' to (2,)

In [76]: optimize.minimize(f, x)
[0. 1.] (2,)
Traceback (most recent call last):
....
  File "<__array_function__ internals>", line 5, in dot
ValueError: shapes (1,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0)

Changing f to work with 1d array:

def f(x):
    print(x, x.shape)
    x = x[:,None]
    Q = np.array([[1,2],[3,4]])
    res = x.T@Q@x
    print(res, res.shape)
    return res.item()
In [86]: f(np.arange(2))
[0 1] (2,)
[[4]] (1, 1)
Out[86]: 4

Now optimize.minimize(f, np.arange(2)) runs (but doesn't converge).