Knowledge MaxxingActivation function
An activation function is a nonlinear transformation applied to a neuron's weighted sum, allowing a network of neurons to represent more than simple linear relationships.
Prerequisite: Artificial neuron
What matters
Without a nonlinear activation function, stacking multiple layers would mathematically collapse into the equivalent of a single linear layer, gaining no real representational power.
Example
A common activation function outputs zero for any negative input and passes positive input through unchanged, letting a network represent sharp, nonlinear patterns.
Common misconception
An activation function is not an optional performance tweak; without any nonlinear activation, a deep network cannot represent more than a single linear model can.
Recall question
Why would stacking layers add no power at all without a nonlinear activation function?