mesuring the performance of system Roc
I am quite new to machine learning and python. Any help would be appreciated.
usually in Matlab, it's easy to plot it. I want to draw the roc curve to evaluate the performance of the face recognition system, i calculate the euclidian distance and the cosine similarity between two images and i would like to apply the computation of its two parameters on a database ( test train). how can I draw the roc curve on this is database images
and how can i measure the performance of autoencoder?
this code doesn't work :
predictions_prob = your_model.predict_proba(x_test)
false_positive_rate, recall, thresholds = roc_curve(y_test, predictions_prob[:,1])
roc_auc = auc(false_positive_rate, recall)
plt.plot(false_positive_rate, recall, 'g', label = 'AUC %s = %0.2f' % ('model name', roc_auc))
plt.plot([0,1], [0,1], 'r--')
plt.legend(loc = 'lower right')
plt.ylabel('Recall')
plt.xlabel('Fall-out')
plt.title('ROC Curve')
this is pre-trained models weights
so now i have two array y_true if two face are similar '1' or if not '0'
y_true [0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
y_score array representy_score
[0.43031937 0.09115553 0.00650781 0.02242869 0.38608587 0.09407699
0.40521139 0.08062053 0.37445426 0.73493853 0.7103999 0.72978038
0.66644344 0.63952136 0.61384821 0.58388719 0.64563826 0.7302449
0.50854671 0.74351138 0.74457312 0.86807218 0.83802608 0.74165669
0.74858481 0.76547028 0.73587325 0.78119443 0.59438175 0.74271324
0.65287331 0.55672997 0.6840947 0.86698833 0.69892132 0.9039218
0.73688647 0.88281097 0.65161654 0.6082072 0.60127196 0.59740826
0.63763261 0.60536379 0.642178 0.61151108 0.62726742 0.61947313
0.67193428 0.7865534 0.65491107 0.6640633 0.68394253 0.63343072
0.79708609 0.78625438 0.70690271 0.75213048 0.76652744 0.85628764
0.82893997 0.75122409 0.76786727 0.7644964 0.75824204 0.78366616
0.65271395 0.75293976 0.72236988 0.56250972 0.72455084 0.9160955
0.74614334 0.94117467 0.75922103 0.91618422]
when i run the code i get this plot:
what should i change scores labels I'm lost any help will appreciate it.
I don't know why i get only 4 elements in tpr and fpr and threshold
fpr [0. 0. 0. 1.]
tpr [0. 0.2 1. 1. ]
thresholds [1.99308544 0.99308544 0.90004301 0. ]
Assuming y_test is a numpy array containing 0 and 1, in which 0 means the two faces are not the same(negative), 1 means the two faces are the same(positive).
Also assuming you use verifyFace in prediction. Let's say it's output is pred, which contains distance between each pairs.
By definition, two faces lower than a threshold will be considered positive. This is just the opposite of typical binary classification task.
So here is a workaround:
from sklearn.metrics import roc_curve, auc
import numpy as np
import matplotlib.pyplot as plt
n_samples = 1000
pred = np.random.randn(n_samples)
y_test = np.random.randint(2, size=(n_samples,))
max_dist = max(pred)
pred = np.array([1-e/max_dist for e in pred])
fpr, tpr, thresholds = roc_curve(y_test, pred)
roc_auc = auc(fpr, tpr)
plt.figure()
lw = 2
plt.plot(fpr, tpr, color='darkorange',
lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic example')
plt.legend(loc="lower right")
plt.show()
The key concept is to convert pred so it looks like a sequence of confidence.
Ref: How to use prediction score in creating ROC curve with Scikit-Learn