Search code examples
rxgboosttidymodels

Tidymodels / XGBoost error in last_fit with rsplit value


I am trying to follow this tutorial here - https://juliasilge.com/blog/xgboost-tune-volleyball/

I am using it on the most recent Tidy Tuesday dataset about great lakes fishing - trying to predict agency based on many other values.

ALL of the code below works except the final row where I get the following error:

> final_res <- last_fit(final_xgb, stock_folds)
Error: Each element of `splits` must be an `rsplit` object.

I searched that error and came to this page - https://github.com/tidymodels/rsample/issues/175 That site has it called a bug and seems to be fixed - but it is with initial_time_split, not initial_split that I am using. I would rather not change it because then I would have to rerun the xgboost that took 9 hours. What went wrong here?

# Setup ----
library(tidyverse)
library(tidymodels)

stocked <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2021/2021-06-08/stocked.csv')

stocked_modeling <- stocked %>% 
  mutate(AGENCY = case_when(
    AGENCY != "OMNR" ~ "other",
    TRUE ~ AGENCY
  )) %>% 
  select(-SID, -MONTH, -DAY, -LATITUDE, -LONGITUDE, -GRID, -STRAIN, -AGEMONTH,
         -MARK_EFF, -TAG_NO, -TAG_RET, -LENGTH, -WEIGHT, - CONDITION, -LOT_CODE,
         -NOTES, - VALIDATION, -LS_MGMT, -STAT_DIST, -ST_SITE, -YEAR_CLASS, -STOCK_METH) %>% 
  mutate_if(is.character, factor) %>% 
  drop_na()

# Start making model ----
set.seed(123)
stock_split <- initial_split(stocked_modeling, strata = AGENCY)
stock_train <- training(stock_split)
stock_test <- testing(stock_split)

xgb_spec <- boost_tree(
  trees = 1000,
  tree_depth = tune(), min_n = tune(), loss_reduction = tune(),
  sample_size = tune(), mtry = tune(),
  learn_rate = tune()
) %>% 
  set_engine("xgboost") %>% 
  set_mode("classification")

xgb_grid <- grid_latin_hypercube(
  tree_depth(),
  min_n(),
  loss_reduction(),
  sample_size = sample_prop(),
  finalize(mtry(), stock_train),
  learn_rate(),
  size = 20
)

xgb_workflow <- workflow() %>% 
  add_formula(AGENCY ~ .) %>% 
  add_model(xgb_spec)

set.seed(123)
stock_folds <- vfold_cv(stock_train, strata = AGENCY)

doParallel::registerDoParallel()


# BEWARE, THIS CODE BELOW TOOK 9 HOURS TO RUN
set.seed(234)
xgb_res <- tune_grid(
  xgb_workflow,
  resamples = stock_folds,
  grid = xgb_grid,
  control = control_grid(save_pred = TRUE)
)

# Explore results
best_auc <- select_best(xgb_res, "roc_auc")

final_xgb <- finalize_workflow(
  xgb_workflow, 
  best_auc)

final_res <- last_fit(final_xgb, stock_folds)

Solution

  • If we look at the documentation of last_fit() We see that split must be

    An rsplit object created from `rsample::initial_split().

    You accidentally passed the cross-validation folds object stock_folds into split but you should have passed rsplit object stock_split instead

    final_res <- last_fit(final_xgb, stock_split)