Search code examples
scikit-learntext-classificationconfusion-matrixperformance-measuring

Why scikit learn confusion matrix is reversed?

I have 3 questions:

1)

The confusion matrix for sklearn is as follows:

TN | FP
FN | TP

While when I'm looking at online resources, I find it like this:

TP | FP
FN | TN

Which one should I consider?

2)

Since the above confusion matrix for scikit learn is different than the one I find in other rescources, in a multiclass confusion matrix, what's the structure will be? I'm looking at this post here: Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative In that post, @lucidv01d had posted a graph to understand the categories for multiclass. is that category the same in scikit learn?

3)

How do you calculate the accuracy of a multiclass? for example, I have this confusion matrix:

[[27  6  0 16]
 [ 5 18  0 21]
 [ 1  3  6  9]
 [ 0  0  0 48]]

In that same post I referred to in question 2, he has written this equation:

Overall accuracy

ACC = (TP+TN)/(TP+FP+FN+TN)

but isn't that just for binary? I mean, for what class do I replace TP with?

Solution

  • As the sklearn guide says: "(Wikipedia and other references may use a different convention for axes)"

    What does it mean? When building the confusion matrix, the first step is to decide where to put predictions and real values (true labels). There are two possibilities:

    • put predictions to the columns, and true labes to rows
    • put predictions to the rows, and true labes to columns

    It is totally subjective to decide which way you want to go. From this picture, Sklearn's Confusion Matrix explained in here, it is clear that scikit-learn's convention is to put predictions to columns, and true labels to rows.

    Thus, according to scikit-learns convention, it means:

    • the first column contains, negative predictions (TN and FN)
    • the second column contains, positive predictions (TP and FP)
    • the first row contains negative labels (TN and FP)
    • the second row contains positive labels (TP and FN)
    • the diagonal contains the number of correctly predicted labels.

    Based on this information I think you will be able to solve part 1 and part 2 of your questions.

    For part 3, you just sum the values in the diagonal and divide by the sum of all elements, which will be

    (27 + 18 + 6 + 48) / (27 + 18 + 6 + 48 + 6 + 16 + 5 + 21 + 1 + 3 + 9)

    or you can just use score() function.