Search code examples
tensorflowkeraslstmloss

ValueError: No gradients provided for any variable when defining custom loss function

underneeth you can find my custom loss function.

def custom_loss_function(y_true, y_pred):
  
  y_pred_bool = tf.math.less_equal(y_pred, tf.constant(0.5))
  y_pred_float = 1 - tf.cast(y_pred_bool, dtype=tf.int32)
  y_true = tf.cast(y_true, dtype=tf.int32)

  mask_bool_loss = tf.math.less(y_true, tf.constant(0))
  mask_loss = 1 - tf.cast(mask_bool_loss, dtype=tf.int32)
  mask = tf.math.reduce_min(mask_loss, axis=2)

  y_multiply = tf.math.multiply(y_true, y_pred_float)

  y = tf.math.reduce_sum(y_multiply, axis=2)
  y_loss = 1 - y
  y_loss = tf.math.multiply(y_loss, mask)

  return y_loss

I know some functions of tensorflow are not differentiable, but I really don't know which functions or how to get around it? Any suggestions for me?

I get this error:

ValueError: No gradients provided for any variable: ['bidirectional_7/forward_lstm_7/lstm_cell_22/kernel:0'/, ...
Solution

  • As soon as you cast your variables to int or bool, all gradient information is lost. So the gradients are broken in this first line.

      y_pred_bool = tf.math.less_equal(y_pred, tf.constant(0.5))

    This is the reason why we usually use things like the binary cross-entropy, as it gives us a differentiable approximation to the non-differentiable 0-1 loss.