I would like to get the average predicted value across CV repeats using caret in R.
require("caret")
data("iris")
fitControl <- trainControl(method = "repeatedcv",
number = 10,
repeats = 10, savePredictions = 'final')
model.cv <- train(Sepal.Length ~ Sepal.Width,
data = iris,
method = "lm",
trControl = fitControl)
head(model.cv$pred)
# intercept pred obs rowIndex Resample
#1 TRUE 5.809386 4.7 3 Fold01.Rep01
#2 TRUE 5.838487 4.6 4 Fold01.Rep01
#3 TRUE 5.460174 5.7 16 Fold01.Rep01
#4 TRUE 5.634780 5.7 19 Fold01.Rep01
#5 TRUE 5.722083 5.2 28 Fold01.Rep01
#6 TRUE 6.071295 4.5 42 Fold01.Rep01
Now I would like to get the average of all the 10 predictions of each example. I can do it by iterating over the examples as follows but I think there has to be a better tidier way.
mean(model.cv$pred[model.cv$pred$rowIndex==1, "pred"])
#[1] 5.745675
EDIT
Following @Obim's answer, I tested the timings of the three proposed solutions. The dplyr
version is way faster. Note that I slightly modified the sapply
version by adding a sort over the unique rowINdex
to keep its output consistent and interpretable.
library("plyr")
library("dplyr")
library("tictoc")
tic("plyr")
for(i in 1:100) meansplyr = ddply(model.cv$pred, ~rowIndex, summarise, mean = mean(pred))
toc()
#plyr: 5.56 sec elapsed
tic("dplyr")
for(i in 1:100) meansdplyr = model.cv$pred %>% group_by(rowIndex) %>% summarise(pred = mean(pred))
toc()
#dplyr: 0.08 sec elapsed
tic("sapply")
for(i in 1:100) {
meanssapply = sapply(
X = sort(unique(model.cv$pred$rowIndex)), # added sort to keep the output consistent
FUN = function(x){mean(model.cv$pred$pred[model.cv$pred$rowIndex %in% x])}
)
}
toc()
#sapply: 0.73 sec elapsed
# the outputs are exactly the same
sum(abs(meansplyr$mean - meansdplyr$pred))
#[1] 0
sum(abs(meansplyr$mean - meanssapply))
#[1] 0
One liner with ddply:
library(plyr)
ddply(model.cv$pred, ~rowIndex, summarise, mean = mean(pred))
Or with dplyr:
library(dplyr)
model.cv$pred %>%
group_by(rowIndex) %>%
summarise(pred = mean(pred))
Another way with sapply (although still iterating over each rowIndex). As @DataD'Oh pointed out, the input should be sorted to allow interpretation of the output:
sapply(
X = sort(unique(model.cv$pred$rowIndex)),
FUN = function(x){mean(model.cv$pred$pred[model.cv$pred$rowIndex %in% x])}
)