Keras model.compile: metrics to be evaluated by the model

Question

I am following some Keras tutorials and I understand the model.compile method creates a model and takes the 'metrics' parameter to define what metrics are used for evaluation during training and testing.

compile(self, optimizer, loss, metrics=[], sample_weight_mode=None)

The tutorials I follow typically use "metrics=['accuracy']". I would like to use other metrics such as fmeasure, and reading https://keras.io/metrics/ I know there is a wide range of options. but I do not know how to pass them to the compile method?

For example:

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['fmeasure'])

will generate an error saying that there is no such metric.

Any suggestions highly appreciated

Thanks

Answer 1

There are two types of metrics that you can provide.
First are the one provided by keras which you can find here which you provide in single quotes like 'mae' or also you can define like

from keras import metrics
model.compile(loss='mean_squared_error',
              optimizer='sgd',
              metrics=[metrics.mae, metrics.categorical_accuracy]) \\or like 
              metrics=['mae', 'categorical_accuracy']

Second is custom metrics like this

import keras.backend as K

def mean_pred(y_true, y_pred):
    return K.mean(y_pred)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy', mean_pred])

Here mean_pred is the custom metric. See the difference in defining the already available metrics and custom defined metrics. So fmeasure is not readily available. You have to define it as custom function.