Knowledge MaxxingGradient boosting
Gradient boosting builds an ensemble of decision trees one at a time, where each new tree is trained specifically to correct the errors of the trees built so far.
Prerequisite: Random forest
What matters
Because each tree targets the previous ensemble's remaining mistakes, gradient boosting often reaches higher accuracy than random forests, at the cost of being more sensitive to overfitting and tuning.
Example
The first tree in a gradient-boosted model might predict roughly, and each following tree focuses on the cases the ensemble so far still gets wrong.
Common misconception
Gradient boosting does not train its trees independently like a random forest; each tree is built specifically in sequence to fix the current ensemble's errors.
Recall question
What does each new tree in gradient boosting specifically try to fix?