Python3.4 scipy.optimize.minimize callbacks multiple times the objective function in one iteration

Question

I am trying to optimize a mathematical model where I use scipy.optimize.minimize to try a set of different input values x in order for my model to return the output values F as close as possible to the target values F_experimental.

In my program, I observed a strange behavior of the scipy.optimize.minimize callback option. The reproducible tester code (that is partially based here) is:

import numpy as np
from scipy.optimize import minimize,rosen, rosen_der

Nfeval = 1

def rosen(X): #Rosenbrock function
  global Nfeval
  print('pass rosen',str(Nfeval))
  #Nfeval += 1
  return (1.0 - X[0])**2 + 100.0 * (X[1] - X[0]**2)**2 + \
         (1.0 - X[1])**2 + 100.0 * (X[2] - X[1]**2)**2

def callbackF(Xi):
  global Nfeval
  print('pass callback',str(Nfeval))
  #print(Nfeval, Xi[0], Xi[1], Xi[2], rosen(Xi))
  Nfeval += 1

x0 = np.array([1.1, 1.1, 1.1])
optXvalues=minimize(rosen, x0, method='Nelder-Mead',callback=callbackF)

Running this code will return on the screen a very strange result, despite the convergence of the minimizer. A part of it, is:

pass rosen 66
pass rosen 66
pass callback 66
pass rosen 67
pass callback 67
pass rosen 68
pass callback 68
pass rosen 69
pass rosen 69
pass callback 69
pass rosen 70
pass rosen 70
pass callback 70
pass rosen 71
pass rosen 71
pass callback 71

The question is why exactly the minimizer passes 2 times through the objective function rosen? And why is not consistent? In iterations 67 and 68 passed only once.

If I deactivate the callback option and add the counter in the rosen function (just un-comment it), the minimizer passes only once.

If I activate a print function (just un-comment it) inside the callbackF function to obtain the values at each iteration, the program passes once more through the rosen function.

The problem is that I need:

1. The minimizer to pass only once through the function because in my real problem, I call a simulator to obtain the results (F) of the proposed optimizer values (x).

2. A way to print/save on each iteration, the iteration number, the x values and the F values.

What do you think is the problem? Can it be a bug in the callback option, a bug in my code or something that I don't understand correctly?

Thanks in advance, Michail

Answer 1

Edit:

The following solution is actually wrong as it changes the way the Nelder-Mead algorithm is supposed to work. I keep it here in case someone else follows this chain of thought but PLEASE IGNORE THE SOLUTION AND GO TO THE COMMENTS.

---

Based on the comments of cel, apparently multiple calls of the objective function is something normal. In optimization problems like mine,however, is obvious that this can not be accepted,because the simulator and post-processor embedded in the objective function, must not rerun.

To circumvent this problem, I found an easy but maybe not robust solution, so please use it with caution.

I added an extra global variable called NfevalPre and in the objective function, I added a condition and a declaration, to check if the function has already been called during this iteration and only if no, to enter it. The final tester code is:

import numpy as np
from scipy.optimize import minimize,rosen, rosen_der

Nfeval = 1
NfevalPre=0

def rosen(X): #Rosenbrock function
  global Nfeval, NfevalPre

  if NfevalPre!=Nfeval:
    print('pass rosen',str(Nfeval))
    NfevalPre=Nfeval

  return (1.0 - X[0])**2 + 100.0 * (X[1] - X[0]**2)**2 + \
         (1.0 - X[1])**2 + 100.0 * (X[2] - X[1]**2)**2

def callbackF(Xi):
  global Nfeval
  print('pass callback',str(Nfeval))
  print(Nfeval, Xi[0], Xi[1], Xi[2], rosen(Xi))
  Nfeval += 1

x0 = np.array([1.1, 1.1, 1.1])
optXvalues=minimize(rosen, x0, method='Nelder-Mead',callback=callbackF,options={'disp':True})

and the results including the extra print/save, are (a part of them):

pass rosen 67
pass callback 67
67 0.999945587739 0.999922683215 0.999847949605 1.08857865253e-07
pass rosen 68
pass callback 68
68 0.999945587739 0.999922683215 0.999847949605 1.08857865253e-07
pass rosen 69
pass callback 69
69 1.00005423035 1.00010796114 1.00019867305 4.44156189226e-08
pass rosen 70
pass callback 70
70 1.00002966558 1.00004883027 1.00010442415 1.88645594463e-08
pass rosen 71
pass callback 71
71 1.00002966558 1.00004883027 1.00010442415 1.88645594463e-08

Of course, I am sure more professional solutions exist. :)