Knowledge Maxxing← Back to topic library
AI MaxxingAdvanced75 sec
Quantisation
Quantisation reduces the numerical precision used to store a model's parameters, shrinking its size and speeding up computation at some cost to accuracy.
Prerequisite: Model serving
What matters
Because lower-precision numbers take less memory and compute to process, quantisation lets a model run on smaller hardware or respond faster, trading a small amount of accuracy for efficiency.
Example
A model whose parameters are stored as 8-bit numbers instead of 32-bit numbers takes roughly a quarter of the memory, at a small, often acceptable accuracy cost.
Common misconception
Quantisation does not change what a model learned; it only changes how precisely its existing parameters are stored and computed.
Recall question
What is quantisation trading away in exchange for a smaller, faster model?
Keywords and related concepts
quantisationprecision reductionmodel compressionknowledge distillation