Search code examples
rrocproc-r-package

Why is the roc function not determining cases and controls properly?

Under the default direction = "auto" setting of the roc function, it appears that cases and controls should be automatically configured such that the ROC curve for a predictor is above the diagonal of the plot and has an AUC >= 0.5.

However, the code shown below incorrectly assigns case and control values, creating an ROC curve whose AUC is < 0.5.

#Creation of data
data <- data.frame("Predictor" = c(rep(0, 59), rep(1, 29)), 
                   "Outcome" = c(rep(0, 40), rep(1, 19), rep(0, 22), rep(1, 7)))

#Fitting ROC
roc_obj <- pROC::roc(Outcome ~ Predictor, data = data)

#Plotting ROC to show that it is below the diagonal
pROC::ggroc(roc_obj) + 
  ggplot2::geom_abline(intercept = 1)

I am aware that I can fix this by manually setting the direction argument in a univariate setting, but I am ultimately attempting to create a multivariable ROC plot, for which I cannot manually provide the direction for each argument.

Is there something I can do to make the default direction = "auto" setting determine my cases and controls properly?

Thanks so much!

Solution

  • Under the default direction = "auto" setting of the roc function, it appears that cases and controls should be automatically configured such that the ROC curve for a predictor is above the diagonal of the plot and has an AUC >= 0.5.

    This is not exactly true. Here is what the document says about direction=auto:

    “auto” (default): automatically define in which group the median is higher and take the direction accordingly. “>”: if the predictor values for the control group are higher than the values of the case group (controls > t >= cases). “<”: if the predictor values for the control group are lower or equal than the values of the case group (controls < t <= cases).

    As you can see, what determines the direction isn't the ROC curve itself, but the medians of the groups. This is usually a reasonable heuristic. But sometimes, when the curve is very close to the diagonal, tiny abnormalities in the distributions can cause the ROC curve to go the other way.

    What are your options?

    1. Accept that your ROC curve is basically the diagonal and its AUC is 0.5.

    2. If that causes numerical issues, you can rebuild the ROC curve if it's below 0.5

      roc_obj <- pROC::roc(Outcome ~ Predictor, data = data, direction = "<") if (auc(roc_obj) < 0.5) { roc_obj <- pROC::roc(Outcome ~ Predictor, data = data, direction = ">") }