Knowledge MaxxingInstruction tuning
Instruction tuning is fine-tuning a model on examples of instructions paired with the desired response, so it learns to follow user requests rather than just continue text.
Prerequisite: Fine-tuning
What matters
Without this step, a raw pretrained model tends to continue a prompt in whatever style resembles its training text, rather than treating it as a request to fulfill.
Example
Instruction tuning teaches a model that given 'Summarize this text', it should produce a summary, not simply continue writing similar text.
Common misconception
Instruction tuning is not the same as preference tuning; it teaches the model to follow instructions at all, while preference tuning refines which of several valid responses is better.
Recall question
What behavior does instruction tuning teach a raw pretrained model to adopt?