How do I apply the binary cross-entropy element-wise and then sum all these losses in Keras?

Question

I want to write a function with two arguments, A and B, tensors of the same shape (for example, 13x13, or some other shape), and that returns a number that represents the summation of all losses when applied binary cross-entropy componentwise. So, for A[i, j] and B[i, j] we find the binary cross-entropy loss, and then sum over all i and j. How to implement that in Keras and Tensorflow?

Answer 1

You can easily define this function using the backend functions sum and binary_crossentropy (or use their equivalents in Tensorflow directly):

def func(A, B):
    return K.sum(K.binary_crossentropy(A,B)) 

Note that K.binary_crossentropy() assumes that the given input values are probabilities; if that's not the case then pass from_logit=True as another argument to it.

Further, if you would like to use this function in a Lambda layer, then you need to change it so that it accepts a list of tensors as input:

def func(inp):
    return K.sum(K.binary_crossentropy(inp[0], inp[1]), [1,2]) # take the sum for each sample independently

# ...
out = Lambda(func)([A, B])

As you can see, [1,2] has been passed to K.sum() as its axis argument to take the sum over all the element of a single sample (and not over the whole batch).