optimizie.minimize using CG

Question

I have been using the following code:

options = {'maxiter':50, 'disp':True}
res = optimize.minimize(
        fun=lrCostFunction,
        x0=theta_k,
        args=(X, y_k, lambda_),
        method='CG',
        options=options
)

And that gives me the following error:

    TypeError: only size-1 arrays can be converted to Python scalars
    The above exception was the direct cause of the following exception:
    ValueError: setting an array element with a sequence.

But when I set jac = True as following:

options = {'maxiter':50, 'disp':True}

res = optimize.minimize(
        fun=lrCostFunction,
        x0=theta_k,
        jac=True,
        args=(X, y_k, lambda_),
        method='CG',
        options=options
)

Everything works fine, but the documentation doesn't state that we must set jac = True, so what's wrong here?

Answer 1

I guess the objective function lrCostFunction returns both function values and the gradients. According to the documentation of minimize

If jac is a Boolean and is True, fun is assumed to return the gradient along with the objective function. If False, the gradient will be estimated using '2-point' finite difference estimation.

Therefore, if jac=False while the objective function returns also the gradient, you will receive some error. For example

# objective function 
def obj_func(x,a,b,c):
    
    func_val = a*x[0]**2 + b*x[1] + c
    func_grad = 2*a*x[0] + b
    return func_val,func_grad

x0 = np.random.rand(3)
res = minimize(obj_func,x0,args=(1,2,3),method='CG')

If I run this code, I will receive a TypeError. If I set jac=True, everything works fine.