Cross Validation and perfcurv in Matlab

Question

I am trying to use perfcurv in a cross validation code. However at some point all the members of the test dataset are of the same class (0). My problem is a binary classification problem. Therefore the following error occurs:

Error using perfcurve (line 368) Less than two classes are found in the array of true class labels.

Error in loadsurfperframe_GIE_leave_one_out (line 669) [x,y,t,AUC(i),OPTROCPT,SUBY,SUBYNAMES] = perfcurve(test_classes,ytest,1);

What should I do to solve this ?

Answer 1

The function perfcurve() computes the Receiver Operating Characteristic (ROC) curve, which shows the trade-off between Sensitivity and 1-Specificity for binary classification problem, i.e. which threshold value is more optimal. If you only have 0-samples is impossible to determine where to place the threshold.

Besides, the specificity is defined as SEN = TP/T = TP/(TP+FN). If we consider negative samples as 0 and positive samples as 1, your data does not have any positive samples. Thus, TP = 0 implies that the value is always SEN=0 independently of the threshold value.