Knowledge MaxxingHyperparameter
A hyperparameter is a setting chosen before training begins, such as learning rate or tree depth, that controls how a model learns rather than being learned from data itself.
Prerequisite: Learning rate
What matters
Unlike a model's learned parameters, hyperparameters must be set or searched over by a person or a tuning process, and a poor choice can prevent even a good algorithm from performing well.
Example
The number of trees in a random forest is a hyperparameter chosen before training, not a value the training process learns on its own.
Common misconception
A hyperparameter is not the same as a model parameter; parameters are learned from data during training, while hyperparameters are fixed choices made beforehand.
Recall question
How does a hyperparameter differ from a parameter the model learns during training?