Knowledge Maxxing← Back to topic library
ML MaxxingIntermediate90 sec
Bias-variance trade-off
The bias-variance trade-off describes how a model's error splits between bias, from being too simple to capture true patterns, and variance, from being too sensitive to noise in the training data.
Prerequisite: Overfitting
What matters
Reducing one side often increases the other, so choosing model complexity is really a choice about where to sit on this trade-off, not a search for a setting with neither problem.
Example
A very simple model tends to have high bias and underfit, while a very complex model tends to have high variance and overfit the same data.
Common misconception
The bias-variance trade-off does not mean more complexity is always worse or always better; the right balance depends on the data and task.
Recall question
Why does reducing a model's bias often increase its variance?
Keywords and related concepts
bias-variance trade-offmodel complexitygeneralisationoverfitting