Knowledge MaxxingLoss function
A loss function measures how far a model's prediction is from the true value on a single example, and training adjusts the model to make this value smaller on average.
Prerequisite: Mean squared error
What matters
The choice of loss function determines exactly what 'better' means during training, so two models trained with different loss functions can end up favoring different kinds of predictions.
Example
A model trained with mean squared error as its loss function will be pushed harder to avoid a few very large errors than one trained with mean absolute error.
Common misconception
A loss function is not the same as an evaluation metric reported to stakeholders; a model can train on one loss function but report a different, more interpretable metric.
Recall question
What determines exactly what 'improvement' means to a model during training?