Knowledge MaxxingAutomation bias
Automation bias is the tendency for a person overseeing a system to over-trust its output and under-scrutinize it, weakening the value of human oversight in practice.
Prerequisite: Human-on-the-loop
What matters
Even a well-designed human-in-the-loop process can fail in practice if the human reviewer routinely defers to the automated suggestion instead of genuinely evaluating it.
Example
A reviewer who always approves a fraud model's flagged transactions without independently checking any of them is displaying automation bias.
Common misconception
Automation bias is not fixed just by requiring a human sign-off step; the human still needs to be genuinely engaged, or the sign-off becomes a rubber stamp.
Recall question
How can automation bias undermine a system that technically has human-in-the-loop oversight?