Gridsearchcv vs Bayesian optimization

Question

Which one among Gridsearchcv and Bayesian optimization works better for optimizing hyper parameters?

Answer 1

There is no better here, they are different approaches.

In Grid Search you try all the possible hyperparameters combinations within some ranges.

In Bayesian you don't try all the combinations, you search along the space of hyperparameters learning as you try them. This enables to avoid trying ALL the combinations.

So the pro of Grid Search is that you are exhaustive and the pro of Bayesian is that you don't need to be, basically if you can in terms of computing power go for Grid Search but if the space to search is too big go for Bayesian.