Ensemble Learning [1]
Ensemble learning, is a model that makes predictions based on a number of different models. By combining a number of different models, an ensemble learning tends to be more flexible (less bias) and less data sensitive (less variance).
The two most popular ensemble learning methods are bagging and boosting.
- Bagging : Training a bunch of models in parallel way. Each model learns from a random subset of the data, where the dataset is same size as original but is randomly sampled with replacement (boostrapped). The application of bagging is found in Random Forests.
- Boosting : Training a bunch of models sequentially. Each model learns from the mistakes of the previous model. That is, the subsequent models tries to explain and predict the error left over by the previous model. The application of boosting is found in Gradient Boosting Decision Trees.
Boosting
Boosting works on the principle of improving mistakes of the previous learner through the next learner.
Motivation
References
[1] https://www.machinelearningplus.com/machine-learning/gradient-boosting/