R: perform car::Anova type II and III test on logistic model built by caret package

Question

I am building a generalized logistic regression model as below:

library(mlbench)
data(PimaIndiansDiabetes)

library(caret)
trControl <- trainControl(method = "repeatedcv",
                          repeats = 3,
                          classProbs = TRUE,
                          number = 10, 
                          savePredictions = TRUE,
                          summaryFunction = twoClassSummary)

caret_model <- train(diabetes~., 
                     data=PimaIndiansDiabetes, 
                     method="glm", 
                     trControl=trControl)

Then I want to perform Anova type II and III test as:

library(car)
car::Anova(caret_model , type=2)

I got the below error:

Error in UseMethod("vcov") : no applicable method for 'vcov' applied to an object of class "c('train', 'train.formula')

However, if I use the function glm to build the model instead, it will be fine:

glm_fit <- glm(diabetes~., data=PimaIndiansDiabetes, family=binomial)
car::Anova(glm_fit, type=2)

So, how do I perform the Anova type II and III test on my caret model?

Answer 1

There is no method anova for a class "train", so to do what you need:

car::Anova(caret_model$finalModel, type=2)
Analysis of Deviance Table (Type II tests)

Response: .outcome
         LR Chisq Df Pr(>Chisq)    
pregnant   15.233  1  9.505e-05 ***
glucose   114.927  1  < 2.2e-16 ***
pressure    6.548  1   0.010502 *  
triceps     0.008  1   0.928500    
insulin     1.742  1   0.186918    
mass       40.779  1  1.704e-10 ***
pedigree   10.340  1   0.001302 ** 
age         2.522  1   0.112253

which is similar to:

car::Anova(glm_fit, type=2)
Analysis of Deviance Table (Type II tests)

Response: diabetes
         LR Chisq Df Pr(>Chisq)    
pregnant   15.233  1  9.505e-05 ***
glucose   114.927  1  < 2.2e-16 ***
pressure    6.548  1   0.010502 *  
triceps     0.008  1   0.928500    
insulin     1.742  1   0.186918    
mass       40.779  1  1.704e-10 ***
pedigree   10.340  1   0.001302 ** 
age         2.522  1   0.112253