Knowledge Maxxing← Back to topic library
Responsible Tech MaxxingIntermediateFairness75 sec
Subgroup evaluation
Subgroup evaluation is measuring a model's performance separately for each demographic group, rather than relying only on one overall aggregate score.
Prerequisite: Fairness trade-off
What matters
An overall score can hide serious underperformance for a smaller group, since strong performance for a majority group can dominate and mask that number.
Example
A model with 95% overall accuracy might still perform at only 70% accuracy for one specific smaller group, a gap invisible without subgroup evaluation.
Common misconception
A strong overall accuracy number does not guarantee consistent performance for every group; it can be an average that hides serious disparities beneath it.
Recall question
Why can a strong overall accuracy score hide poor performance for a specific subgroup?
Keywords and related concepts
subgroup evaluationdisaggregated metricsfairness testingfairness trade-off