Using python scipy.optimize.minimize with function that evaluates both value and gradient

Question

I want to find local minima of a function (of several variables) using python. The set of gradient-based optimization methods described in scipy.optimize.minimize seem like a good way to start.

I can compute the value of the function as well as the gradient of this function. As a matter of fact, when I evaluate the function, I basically get the gradient for free. Is there a way to leverage this property to minimize the number of function calls using scipy.optimize.minimize ?

I'm only referring to methods that use gradient based optimization (say BFGS for instance).

More precisely, how can I plug in a single python function that computes both the value of my mathematical function and the value of its gradient into scipy.optimize.minimize ?

Instead of this :

res = minimize(fun, x0, method='BFGS', jac=grad_fun,options={'disp': True})

I would like something like this :

res = minimize(fun_and_grad, x0, method='BFGS', options={'disp': True})

Thank you !

Answer 1

You can use a custom class that caches the gradient and then returns it when requested:

class Wrapper:
    def __init__(self):
        self.cache = {}

    def __call__(self, x, *args):
        fun, grad = compute_something(x)
        self.cache['grad'] = grad
        return fun

    def jac(self, x, *args):
        return self.cache.pop('grad')


wrapper = Wrapper()
res = minimize(wrapper, x0, jac=wrapper.jac, method='BFGS', options={'disp': True})