scikit-learn classification report - Precision and F-score are ill-defined and being set to 0.0

Question

I am using scikit-learn to compute classification record, when I try to print it, it gives a warning :

print classification_report(original,predicted)

Warning:

Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.

So I want to report precision and recall via micro averaging. How can I use classification report so that it results in not ill-defined precision and recall ? Thanks in advance.

Answer 1

It's not a matter of the classification report itself; reading closely, the warning in fact tells you that some of the classes are absent from your predictions (i.e. your classifier has predicted no samples as belonging to some classes).

Adapting the example from the docs helps demonstrating this:

from sklearn.metrics import classification_report
y_true = [0, 1, 2, 2, 0]
y_pred = [0, 0, 2, 2, 0] # no 1's predicted
target_names = ['class 0', 'class 1', 'class 2']

print classification_report(y_true, y_pred, target_names=target_names)

Here is the result:

             precision    recall  f1-score   support

    class 0       0.67      1.00      0.80         2
    class 1       0.00      0.00      0.00         1
    class 2       1.00      1.00      1.00         2

avg / total       0.67      0.80      0.72         5

/home/ctsats/.local/lib/python2.7/site-packages/sklearn/metrics/classification.py:1135: 
UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
  

So, the issue is with your classifier and/or training data...

UPDATE (after comments): By definition, only the macro-average gives an F1 score for each class (label); from the docs:

'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.

'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

This is the reason why in classification_report the F1 score reported is via macro-average. It is true that using micro-average you don't get a warning (since it counts the total true positives, hence it doesn't care if some classes are not represented in the predictions), but you don't get the F1 score per class either - you only get a single, overall number:

f1_score(y_true, y_pred, average='micro')
# 0.8000000000000002

which cannot be used in the classification_report, where all reported metrics are per class.