Knowledge Maxxing← Back to topic library
Responsible Tech MaxxingIntermediateBias75 sec
Sampling bias
Sampling bias occurs when the way data is collected systematically over- or under-represents certain groups compared to the real population the model will be used on.
Prerequisite: Dataset bias
What matters
A model trained on a non-representative sample can perform well in evaluation yet fail on the real population it's actually deployed against, if that population differs from the sample.
Example
Collecting survey data only from smartphone app users systematically excludes people without smartphones, skewing the sample away from the full population.
Common misconception
Sampling bias is not the same as simply having a small dataset; a large dataset can still be badly biased if it was collected in a systematically skewed way.
Recall question
How can a large dataset still suffer from sampling bias?
Keywords and related concepts
sampling biasrepresentative datacollection methoddataset bias