using `switch`/`cond` in a custom loss function

Question

I need to implement a custom loss function in keras that computes the standard categorical crossentropy except when the y_true is all zeros.

This is my attempt to do so:

def masked_crossent(y_true, y_pred):
    return K.switch(K.any(y_true),
                    losses.categorical_crossentropy(y_true, y_pred),
                    losses.categorical_crossentropy(y_true, y_pred) * 0)

However, I get the following error once training starts (compilation works fine):

~/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in init(self, graph, fetches, feeds) 419 self._ops.append(True) 420 else: --> 421 self._assert_fetchable(graph, fetch.op) 422 self._fetches.append(fetch_name) 423 self._ops.append(False)

~/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _assert_fetchable(self, graph, op) 432 if not graph.is_fetchable(op): 433 raise ValueError( --> 434 'Operation %r has been marked as not fetchable.' % op.name) 435 436 def fetches(self):

ValueError: Operation 'IsVariableInitialized_4547' has been marked as not fetchable.

In place of losses.categorical_crossentropy(y_true, y_pred) * 0, I've also tried the following with various other errors (either during compilation or once training has started):

                    K.zeros_like(losses.categorical_crossentropy(y_true, y_pred))

                    K.zeros((K.int_shape(y_true)[0]))

                    K.zeros((K.int_shape(y_true)[0], 1))

... though I imagine that there is a trivial way to do this.

Answer 1

I only have an idea for a workaround:

def masked_crossent(y_true, y_pred):
    return K.max( y_true ) * K.categorical_crossentropy(y_true, y_pred)

You need to add the axis = -1 if this is for whole batches.