I calculate the AUROC (AUC for the ROC curve) first using the ROCR package and then manually (using my get_au_curve()
function) as below.
Unfortunately, the two results do not agree. I am assuming ROCR's result is the correct one. Is the issue here with my function approximator?
rm(list=ls())
if(!require("ROCR")) { install.packages("ROCR"); require("ROCR") }
# Function to return area under the curve for ROC or PR curves
get_au_curve <- function(x, y) {
pr_perf <- performance(pred, measure=y, x.measure=x )
x_list <- pr_perf@x.values[[1]]
y_list <- pr_perf@y.values[[1]]
if (y == "prec") { # if it is an Area under PR curve, impute precision[1], whcih is NaN, with 1
y_list[is.na(y_list)] <-1 }
f_appr <- approxfun( cbind(x_list, y_list) ) # function approximator for prediction-recall or ROC curve
auc <- integrate(f_appr, 0, 1)
return(auc$value)
}
predictions <- c(0.61, 0.36, 0.43, 0.14, 0.38, 0.24, 0.97, 0.89, 0.78, 0.86)
labels <- c(1, 1, 1, 0, 0, 1, 1, 1, 0, 1)
pred <- prediction(predictions, labels)
# AUROC
# 1 Using ROCR
perf2 <- performance(pred, "auc")
auroc<- perf2@y.values
# 2. Using the function I wrote
auroc_manual <- get_au_curve('fpr', 'tpr')
This gives the result:
> auroc_manual
[1] 0.6785714
> auroc
[[1]]
[1] 0.7142857
The approxfun
is not appropriate to compute a ROC curve. The tied values in x
are averaged and an interpolation between x
is calculated. Compare:
plot(x_list, y_list, type="l")
curve(f_appr)
You should use caTools::trapz or a similar function that calculates the AUC with the trapezoidal rule.