Knowledge MaxxingBackpropagation
Backpropagation is the algorithm that efficiently computes how much each parameter in a neural network contributed to its overall error, so gradient descent knows how to adjust each one.
Prerequisite: Activation function
What matters
It works backward from the output layer to the input layer, reusing intermediate calculations, which is what makes training deep networks with millions of parameters computationally practical.
Example
After a network makes a wrong prediction, backpropagation calculates exactly how much each individual weight contributed to that error, layer by layer, from output back to input.
Common misconception
Backpropagation is not a separate training algorithm from gradient descent; it is the method used to compute the gradients that gradient descent then uses to update parameters.
Recall question
In which direction does backpropagation compute a network's error contributions?