SciPy Optimizer gives result that do not satisfy constraints

Question

I am using scipy.optimize.minimize to solve a problem, but the result given by the package violates the constraint.

The case is mad simple where only one objective function and only one constraint are given. Here is the code:

   import math
   import numpy as np
   import scipy
   from scipy.optimize import minimize

   def objective(x):
       return np.sum(np.dot(x,x))
   n = 5
   X_bound=[(0,4) for i in range(n)]
   X_guess=[1 for i in range(n)]
   _tmp = []
  func_list = []

  def temp_func(X):
      total = 0
      for i in range(n):
          total = total + np.maximum(X[i] * 5 - 6, 0)
      return total/n - 1
  func_list.append(temp_func)
  for ii in range(len(func_list)):
  _tmp.append({'type': 'ineq', 'fun': func_list[ii]})
  X_constraint=_tmp

  sol=scipy.optimize.minimize(objective,X_guess,method='SLSQP',bounds=X_bound,
                        constraints=X_constraint)
  result = sol.x
  result

The result given by the above code is

array([5.61582806e-12, 3.56925226e-12, 3.57912934e-12, 3.57912933e-12, 3.57872619e-12])

which obviously violates the one (and only) constraint.

Any ideas what i do wrong? Thanks

Answer 1

The optimization isn't succeeding. You can see the solution reports failure... at least it knows it wasn't able to satisfy the constraint.

>>> result
array([5.61582806e-12, 3.56925226e-12, 3.57912934e-12, 3.57912933e-12,
       3.57872619e-12])
>>> temp_func(result)
-1.0
>>> sol
     fun: 8.270470124157974e-23
     jac: array([1.49123928e-08, 1.49082997e-08, 1.49083195e-08, 1.49083195e-08,
       1.49083186e-08])
 message: 'Iteration limit exceeded'
    nfev: 1697
     nit: 101
    njev: 101
  status: 9
 success: False
       x: array([5.61582806e-12, 3.56925226e-12, 3.57912934e-12, 3.57912933e-12,
       3.57872619e-12])

The cause for the optimization failure is the maximum function in the constraint. Sequential quadratic programming is based on linearizing the constraints. Locally, the (unsatisfied) constrain linearizes to a constant value so the iterative solution doesn't see any way to make progress satisfying that constraint -- each iteration reduces the objective without making progress toward the solution.

I changed the constraint to not be flat in regions to allow the algorithm to incrementally improve it and the optimization succeeds.

def temp_func(X):
  total = 0
  for i in range(n):
      val = total+X[i]*5 - 6
      if val < 0:
        val = np.arctan(val)
      total = val
  return total/n - 1

That probably the constraint that you are actually trying to enforce but perhaps you can smoothly approximate your constraint and get something that succeeds.