Terminate scipy minimize as soon as function values is below threshold

Question

I have a function of some parameters that will return a probability. How can I set scipy's minimize to terminate as soon as finds some parameters that will return a probability below a certain threshold (even if it is a "large" probability like 0.1 or so)?

Thanks a lot!

Answer 1

The first answer is: it depends on the underlying solver you use. Most of the time, SciPy is just wrapping around efficient implementations in other languages (e.g. SLSQP in Fortran).

This is not the case for trust-constr, which is implemented in Python, and allows a callback returning True to stop the optimization process. See the documentation of the callback argument of scipy.optimize.minimize for more details.

For other solvers, the most straightforward way to achieve what you want is by implementing your own exception, similar to what suggested Andrew Nelson. You will not be able to get the inner state of the solver, but your Python script can go on, and the function is evaluated only once at each candidate point.

Here is a reproducible example using the Nelder-Mead Simplex Downhill algorithm:

from scipy.optimize import minimize
from numpy import inf


class Trigger(Exception):
    pass


class ObjectiveFunctionWrapper:

    def __init__(self, fun, fun_tol=None):
        self.fun = fun
        self.best_x = None
        self.best_f = inf
        self.fun_tol = fun_tol or -inf
        self.number_of_f_evals = 0

    def __call__(self, x):
        _f = self.fun(x)

        self.number_of_f_evals += 1

        if _f < self.best_f:
            self.best_x, self.best_f = x, _f

        return _f

    def stop(self, *args):
        if self.best_f < self.fun_tol:
            raise Trigger


if __name__ == "__main__":

    def f(x):
        return sum([xi**2 for xi in x])

    fun_tol = 1e-4
    f_wrapped = ObjectiveFunctionWrapper(f, fun_tol)

    try:
        minimize(
            f_wrapped,
            [10] * 5,  # problem dimension is 5, x0 is [1, ..., 1],
            method="Nelder-Mead",
            callback=f_wrapped.stop
        )
    except Trigger:
        print(f"Found f value below tolerance of {fun_tol}\
            in {f_wrapped.number_of_f_evals} f-evals:\
            \nx = {f_wrapped.best_x}\
            \nf(x) = {f_wrapped.best_f}")
    except Exception as e:  # catch other errors
        raise e

Output:

Found f value below tolerance of 0.0001            in 239 f-evals:            
x = [ 0.00335493  0.00823628 -0.00356564 -0.00126547  0.00158183]            
f(x) = 9.590933918640515e-05