Knowledge MaxxingGradient descent
Gradient descent is an optimization method that repeatedly adjusts a model's parameters in the direction that most reduces the loss function.
Prerequisite: Loss function
What matters
This is the core mechanism behind training most modern models: each step nudges parameters slightly, gradually walking the loss toward a lower value rather than solving for the optimum directly.
Example
Training a model might take thousands of gradient descent steps, each slightly adjusting parameters to make predictions a little more accurate than the step before.
Common misconception
Gradient descent does not guarantee finding the single best possible set of parameters; it can settle into a local minimum that is good but not globally optimal.
Recall question
What does each single step of gradient descent do to a model's parameters?