I'm trying to add multiple evaluation metrics to an XGBoost training job using Sagemaker, documentation says it is possible (https://github.com/dmlc/xgboost/blob/master/doc/parameter.rst#learning-task-parameters):
User can add multiple evaluation metrics. Python users: remember to pass the metrics in as list of parameters pairs instead of map, so that latter eval_metric won't override previous one
The documentation hasn't any code examples But I have tried many ways to do it(including the simple passing them as a list, ex: eval_metric=['mae', 'merror']), but I just cannot find a syntax that works. Any hints?
Sagemaker SDK implementation of XGBOOST follows dmlc/xgboost. So you can just pass the eval_metric as you would with xgboost.ai.
xgb_model.fit(trainData, targetVar, early_stopping_rounds=10,
eval_metric=['mae', 'merror', 'aucpr'], eval_set=[(valData, valTarget)])
In the example above, we are passing three evaluation metrics. However, if you are trying to pass custom metrics then the above implementation wouldn't work.