Knowledge MaxxingSafety evaluation
Safety evaluation is systematically checking a system against a defined set of harm categories before deployment, to confirm it behaves acceptably across each of them.
Prerequisite: Adversarial testing
What matters
Having a defined set of categories to check against ensures safety testing is systematic and repeatable, rather than relying on whatever issues happen to be noticed informally.
Example
A safety evaluation might specifically check a model's responses across categories like self-harm content, dangerous instructions, and harassment, one by one.
Common misconception
A safety evaluation passing once is not a permanent guarantee; new failure modes can emerge as a model is updated or used in new contexts over time.
Recall question
Why does having defined harm categories make safety evaluation more reliable than informal review?