Shape error only when TPU training Keras model

Question

First off, this is not my code. I just changed it to be able to train it on TPU. The original author is here. I am able to run it on the GPU accelerated runtime on a collaboratory notebook but it seems to break when I do TPU accelerated runtime.

Here is my notebook. It just give me an error that the activation function is not the right size.

ValueError: Error when checking target: expected activation_21 to have shape (1,) but got array with shape (205,)

I would appreciate any help I can get as I spent like 3 hours debugging.

Answer 1

Since you are one-hot encoding the labels and therefore they are not sparse, you need to use 'categorical_accuracy' as the metric:

model.compile(..., metrics=['categorical_accuracy'])

or more succinctly use 'accuracy' to let Keras infer the right metric based on the loss function used (which in this case would be 'categorical_accuracy' since you are using categorical_crossentropy as the loss function):

model.compile(..., metrics=['accuracy'])