Knowledge Maxxing← Back to topic library
AI MaxxingIntermediate75 sec
Pretraining
Pretraining is the initial stage where a model learns general language patterns from a huge, broad dataset, before any task-specific adjustment happens.
Prerequisite: Large language model
What matters
This stage is what gives a model its broad knowledge and fluency; later stages like fine-tuning adjust behavior but rely entirely on what pretraining already established.
Example
A model pretrained on a large snapshot of internet text learns grammar, facts and reasoning patterns before ever seeing a task-specific dataset.
Common misconception
Pretraining is not the same as fine-tuning; pretraining builds general capability, while fine-tuning narrows behavior toward a specific use case.
Recall question
What capability does pretraining give a model before any fine-tuning happens?
Keywords and related concepts
pretrainingtraining datageneral capabilityfine-tuning