Knowledge MaxxingData leakage
Data leakage happens when information that would not be available at real prediction time accidentally influences training, making evaluation results look better than they truly are.
Prerequisite: Test set
What matters
Leakage is dangerous because it is often invisible until deployment, when the model performs far worse in production than its evaluation numbers suggested.
Example
Including a customer's cancellation date as a feature when predicting churn is leakage, since that date would not be known before the churn actually happens.
Common misconception
Data leakage is not the same as a model simply performing poorly; leakage specifically makes evaluation numbers look artificially good, hiding a real problem.
Recall question
Why can a model with leaked data show great evaluation results but fail in production?