Search code examples
rr-caretrocauc

How to compute area under ROC curve from predicted class probabilities, in R using pROC or ROCR package?

I have used caret library to compute class probabilities and predictions for a binary classification problem, using 10-fold cross validation and 5 times repetition.

Now I have TRUE (observed values for each data point) values, PREDICTED (by an algorithm) values, Class 0 probabilities and Class 1 probabilities which were used by an algorithm to predict class label.

Now how can I create an roc object using either ROCR or pROC library and then calculate auc value?

Assume that I have all these values stored in predictions dataframe. e.g. predictions$pred and predictions$obs are the predicted and true values respectively, and so on...

Solution

  • Since you did not provide a reproducible example, I'm assuming you have a binary classification problem and you predict on Class that are either Good or Bad.

    predictions <- predict(object=model, test[,predictors], type='prob')

    You can do:

    > pROC::roc(ifelse(test[,"Class"] == "Good", 1, 0), predictions[[2]])$auc
# Area under the curve: 0.8905