Knowledge MaxxingOverfitting
Overfitting happens when a model learns patterns specific to its training data, including its noise, so closely that it performs worse on new, unseen data.
Prerequisite: Generalisation
What matters
A large gap between strong training performance and weak validation performance is the clearest sign of overfitting, and it is one of the most common failure modes in practice.
Example
A model that memorizes exact training examples and their outlier quirks, rather than the general pattern, will do poorly on any new example that varies slightly.
Common misconception
Overfitting is not fixed by simply adding more model complexity; it is more complexity, more training time, or too little data that usually causes it in the first place.
Recall question
What gap between training and validation performance signals overfitting?