Knowledge Maxxing← Back to topic library
AI MaxxingIntermediate75 sec
Human evaluation
Human evaluation is judging model output quality by having people directly review and rate responses, rather than relying only on automated scoring.
Prerequisite: AI benchmark
What matters
Many qualities that matter, like helpfulness, tone, and honesty, are hard to capture in an automated metric, so human judgment remains necessary alongside benchmarks for a fuller picture.
Example
A company might have reviewers rate 200 chatbot responses for helpfulness and politeness, since no automated score reliably captures both at once.
Common misconception
Human evaluation is not automatically more objective than automated metrics; different human raters can disagree, so results depend on who is rating and by what criteria.
Recall question
Why is human evaluation still used alongside automated benchmarks?
Keywords and related concepts
human evaluationquality ratingautomated metricsai benchmark