Search code examples
xgboostmlr3

How to set "budget" tag for xgboost hyperband optimization with mlr3tuningspaces?


I am trying to tune xgboost with hyperband and I would like to use the suggested default tuning space from the mlr3tuningspaces package. However, I don't find how to tag a hyperparameter with "budget" while using lts .

Below, I reproduced the mlr3hyperband package example to illustrate my issue:

library(mlr3verse)
library(mlr3hyperband)
library(mlr3tuningspaces)

## this does not work, because I don't know how to tag a hyperparameter 
## with "budget" while using the suggested tuning space
search_space = lts("classif.xgboost.default")
search_space$values

## this works because it has a hyperparameter (nrounds) tagged with "bugdget"
search_space = ps(
  nrounds = p_int(lower = 1, upper = 16, tags = "budget"), 
  eta = p_dbl(lower = 0, upper = 1),
  booster = p_fct(levels = c("gbtree", "gblinear", "dart"))
)

# hyperparameter tuning on the pima indians diabetes data set
instance = tune(
  method = "hyperband",
  task = tsk("pima"),
  learner = lrn("classif.xgboost", eval_metric = "logloss"),
  resampling = rsmp("cv", folds = 3),
  measures = msr("classif.ce"),
  search_space = search_space,
  term_evals = 100
)

# best performing hyperparameter configuration
instance$result

Solution

  • Thanks for pointing this out. I will add the budget tag to the default search space. Until then you can use this code.

    library(mlr3hyperband)
    library(mlr3tuningspaces)
    library(mlr3learners)
    
    # get learner with search space in one go
    learner = lts(lrn("classif.xgboost"))
    
    # overwrite nrounds with budget tag
    learner$param_set$values$nrounds = to_tune(p_int(1000, 5000, tags = "budget"))
    
    instance = tune(
      method = "hyperband",
      task = tsk("pima"),
      learner = learner,
      resampling = rsmp("cv", folds = 3),
      measures = msr("classif.ce"),
      term_evals = 100
    )
    

    Update 28.06.2022

    The new API in version 0.3.0 is

    learner = lts(lrn("classif.xgboost"), nrounds = to_tune(p_int(1000, 5000, tags = "budget"))