glmnet variable importance | `vip` vs `varImp`

Question

I fitted a Lasso Logistic Regression model using glmnet

X = data.matrix(mtcars %>% dplyr::select(-vs))
y = mtcars$vs
CV_lasso = cv.glmnet(X, y, alpha = 1, family = "binomial", type.measure = "class")

I was trying to determine variable importance and came across two methods that give widely different results

vip(CV_lasso$glmnet.fit)

varImp(CV_lasso$glmnet.fit, lambda = CV_lasso$lambda.min) 

Could someone tell me what the difference is with these methods and which one is better? What is the best way to determine variable importance from a logistic lasso model?

Answer 1

It seems to be because they use a different value for lambda. In varImp, you specify which lambda to use. However, when you call vip, then in the source code, the following code gets executed:

imp <- if (inherits(object, what = "vi")) {
    object
  } else {
    vi(object = object, ...)  # compute variable importance scores
  }

where vi(object = object, ...) gets called. In the source code for the vi function, at some point this tibble is defined:

tib <- switch(method,
    "model" = vi_model(object, ...),
    "firm" = vi_firm(object, feature_names = feature_names, var_fun = var_fun,
                     ice = ice, ...),
    "permute" = vi_permute(object, feature_names = feature_names, ...),
    vi_shap(object, feature_names = feature_names, ...)
  )

The problem arises with the definition of "model". When trying to run vi_model. In this function this code is what finds out which lambda to use:

 # Extract coefficients
  s <- list(...)$s
  if (is.null(s)) {
    s <- min(object$lambda)
  }

Since you haven't defined what s, then it is set to be the minimum of you lambdas. That is the last value of lambda, which is 0.0001119218. This is not equal to the lambda used in your varImp, where you specify to use the optimal lambda, in this case 0.1586463.

You can get the same coefficients by running:

c1 = vip::vi_model(CV_lasso$glmnet.fit, s = CV_lasso$lambda.min)
c2 = caret::varImp(CV_lasso$glmnet.fit, lambda = CV_lasso$lambda.min) 

and then if you insist on using vip::vip to get the plot, you can call this function on c1:

vip::vip(c1)