Apache commons math optimization get best solution each iteration

Question

Apache Commons Math has optimization classes that can optimize some objective function, and typically a maximum number of evaluations has to be set for an optimizer.

In https://stackoverflow.com/a/32532514/3229995 there is a working small Java example of a full functional NelderMeadSimplex with MaxEval(100) setting the maximal evaluations.

How can I obtain the current solution during each evaluation step of an optimizer such as NelderMeadSimplex?

The code referenced runs and prints out only the final solution (or throws a TooManyEvaluationsException and does not print any solution so far if the number in MaxEval(100) is too low), but I cannot find a class method or field that can help to obtain the current solution of each eval step.

Answer 1

As far as I know, and as far as I could derive from looking at the source code of the implementation, there is no direct, built-in way to obtain the intermediate results.

There are some interfaces involved that are (supposed to be) called during each evaluation. Particularly, there is the ConvergenceChecker interface. One comparatively clean solution could be to implement this interface accordingly (by deriving from one of its implementing classes). But as far as I know, there is no strict requirement for each optimization algorithm to use this interface (and even less to reall call it during each evaluation step).

Once, I wanted to achieve the same goal: To "log" the intermediate steps of the optimization, to visualize an optimization process. I solved this ... very pragmatically: The only thing that one can be sure of is that the MultivariateFunction will have to be evaluated at each point. This is, in some sense, the place where one can insert a hook into the optimization black box. So eventually, I created a wrapper around the MultivariateFunction:

private static MultivariateFunction createLogging(
    final MultivariateFunction delegate)
{
    return new MultivariateFunction()
    {
        @Override
        public double value(double[] point)
        {
            double result = delegate.value(point);
            System.out.println("Evaluated "+
                Arrays.toString(point)+" to "+result);
            return result;
        }
    };
}

Then, replacing any

new ObjectiveFunction(multivariateFunction)

with

new ObjectiveFunction(createLogging(multivariateFunction)),

will cause the optimization process to be logged.

I do not recommend to solve it like this, but depending on the goal and application case, it may be an option, at least.