Search code examples
adaboost

Issue in understanding a part of Adaboost theory


Source:- https://machinelearningmastery.com/boosting-and-adaboost-for-machine-learning/

AdaBoost can be used to boost the performance of any machine learning algorithm. It is best used with weak learners. These are models that achieve accuracy just above random chance on a classification problem.

I did not understand what the highlighted( bold and italic ) part of the above text is trying to say. Can someone kindly explain it?


Solution

  • Consider a two-class problem, performance based on chance alone is 0.5 (1/2). So, you need to select a weak classifier that is right greater than or equal to half the times.

    Let us say you have some classifier that can give you a performance of 0.51. You follow the steps as in the article you have read already, and with the addition of each weak classifier, the performance improves.

    The reason why they mention it is best used with weak learners is that you get the highest 'benefit' from that, in terms of computational complexity and performance tradeoff from a practical view point. If you already had a classifier that was say 0.9 accuracy, then, the gain out of boosting would not be as much as starting with a classifier that had say 0.51.