Search code examples
pythonnumpymathcross-entropy

python - invalid value encountered in log

I have the following expression: log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))

it is giving me the following warning:

RuntimeWarning: invalid value encountered in log
  log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))

I don't understand what might be the invalid value or why am I getting it. Any and every help is appreciated.

NOTE: This is a cross-entropy cost function where I added 1e-7 to avoid having zeros inside log. y & a are numpy arrays and numpy is imported as np.

Solution

  • You probably still have negative values inside the log, which gives nan with real numbers.

    a and y should represent probability between 0 to 1, So you need to check why do you have smaller/larger values there. Adding 1e-7 shows there is something fishy, because np.log(0) gives -inf, which I think is the value you want.