Python Scipy Optimization: Runtime Warning Invalid Value Encountered in Log

Question

I am trying to optimize the first function given the latter two constraining function, using Scipy.

def entropy(x):
    entropy = 0
    for i in range(6):
        entropy = entropy + x[i]*np.log(x[i])
    return entropy

def constraint_1(x):
    validpmf = 0
    for i in range(6):
        validpmf = validpmf + x[i]
    return validpmf - 1

def constraint_2(x):
    mean = 0
    for i in range(7):
        mean = mean + (i*x[i-1])
    return mean - 4.5

Here is the Scipy code.

ans = sp.optimize.minimize(entropy, [.04,.08,.1,.15,.25,.35], \
                           constraints = cons, jac = False, method = 'SLSQP')

I am receiving the actual correct answer back, but I am getting a runtime warning:

[ 0.05447023  0.07863089  0.1140969   0.16556351  0.23970755  0.34753092]
RuntimeWarning: invalid value encountered in log
entropy = entropy + x[i]*np.log(x[i])

I had this issue before with a simpler optimization problem where it was returning an incorrect answer which I fixed by changing my initial guesses. I don't understand why that had worked. In this case however, the initial guesses are quite good approximations so I want to keep them, and also changing them around hasn't managed to mitigate the runtime warning.

To summarize, solution is correct but I don't understand the runtime warning.

Answer 1

The log is not defined at zero, or if you are a computer around zero.

You can suppress the warning by restricting to positive values:

import numpy as np
import scipy as sp
import scipy.optimize

def entropy(x):
    entropy = 0
    for i in range(6):
        entropy = entropy + x[i]*np.log(x[i])
    return entropy

def constraint_1(x):
    validpmf = 0
    for i in range(6):
        validpmf = validpmf + x[i]
    return validpmf - 1

def constraint_2(x):
    mean = 0
    for i in range(6):
        mean = mean + (i*x[i])
    return mean - 4.5

abstol = 1e-6

cons = ({'type': 'eq', 'fun': constraint_1},
        {'type': 'eq', 'fun': constraint_2},
        {'type': 'ineq', 'fun': lambda x: x - abstol})


ans = sp.optimize.minimize(entropy, [.04,.08,.1,.15,.25,.35], \
                           constraints = cons, jac = False, method = 'SLSQP')