Search code examples
pythonscikit-learncross-validationscoring

Multi-scoring input RandomizedSearchCV

I know you can input multiple scorers when performing RandomizedSearchCV but I couldn't find which one will then be used for optimisation.

scoring = {'Log loss': 'neg_log_loss', 'AUC': 'roc_auc', 'F1': 'f1', 'Bal Acc': 'balanced_accuracy'}

search_RF = RandomizedSearchCV(RF_model, parameters_RF, scoring = scoring, 
                                n_jobs = -1, cv = cv_RSKFCV, n_iter = 200,
                                random_state = 2504).fit(X_train, y_train)

In the above example, will it then optimise the 'neg_log_loss'?

Solution

  • It optimises all of them, taking into consideration one at a time. You can check the results for all of them in search_RF.cv_results_.

    Also you should use refit parameter, instead of keeping it to default as you will get an error if you will try running search_RF.best_estimator_.

    Follow below links for more details:

    https://scikit-learn.org/stable/auto_examples/model_selection/plot_multi_metric_evaluation.html#sphx-glr-auto-examples-model-selection-plot-multi-metric-evaluation-py

    https://scikit-learn.org/stable/modules/grid_search.html#multimetric-grid-search