Search code examples
rrocauc

ROCR package AUROC output is different from manual calculation


I calculate the AUROC (AUC for the ROC curve) first using the ROCR package and then manually (using my get_au_curve() function) as below.

Unfortunately, the two results do not agree. I am assuming ROCR's result is the correct one. Is the issue here with my function approximator?

rm(list=ls())
if(!require("ROCR")) { install.packages("ROCR");  require("ROCR") }

# Function to return area under the curve for ROC or PR curves
get_au_curve <- function(x, y)  {

  pr_perf <- performance(pred, measure=y, x.measure=x )
  x_list <- pr_perf@x.values[[1]]
  y_list <- pr_perf@y.values[[1]]

  if (y == "prec") { # if it is an Area under PR curve, impute precision[1], whcih is NaN, with 1
    y_list[is.na(y_list)] <-1  }

  f_appr <- approxfun( cbind(x_list, y_list) ) # function approximator for prediction-recall or ROC curve
  auc <- integrate(f_appr, 0, 1)

  return(auc$value)
}

predictions <- c(0.61, 0.36, 0.43, 0.14, 0.38, 0.24, 0.97, 0.89, 0.78, 0.86)
labels      <- c(1,    1,    1,    0,    0,     1,    1,    1,    0,     1)

pred <- prediction(predictions, labels)

# AUROC
# 1 Using ROCR
perf2 <- performance(pred, "auc")
auroc<- perf2@y.values

# 2. Using the function I wrote
auroc_manual <- get_au_curve('fpr', 'tpr')

This gives the result:

> auroc_manual
[1] 0.6785714
> auroc
[[1]]
[1] 0.7142857

Solution

  • The approxfun is not appropriate to compute a ROC curve. The tied values in x are averaged and an interpolation between x is calculated. Compare:

    plot(x_list, y_list, type="l")
    curve(f_appr)
    

    You should use caTools::trapz or a similar function that calculates the AUC with the trapezoidal rule.