Knowledge MaxxingExplainability
Explainability is a system's ability to provide a human-understandable reason for a specific decision it made, even if that explanation is generated after the fact.
Prerequisite: Transparency
What matters
A good explanation helps an affected person or reviewer understand why a specific outcome occurred, which is different from understanding exactly how the whole model works internally.
Example
Telling a denied loan applicant that 'insufficient credit history' was the main factor is an explanation, even if it doesn't reveal the model's full internal computation.
Common misconception
Explainability is not the same as interpretability; an explanation can be a plausible, useful reason without being a complete, literal description of the model's internal reasoning.
Recall question
Why can an explanation be useful without describing a model's full internal reasoning?