Research area 02
Explainability and interpretability
Methods that make model behaviour legible to the people who depend on it, from feature attribution to mechanistic analysis, and the evidence that an explanation is faithful.
- xai
- interpretability
- attribution
- human-factors
An AI system that cannot be understood cannot be audited, debugged, or trusted with consequential decisions. Explainable AI (XAI) is the set of methods that try to close that gap: feature attribution, counterfactual explanations, concept-based methods, and the newer mechanistic work that studies the internal structure of neural networks directly.
We approach explainability as an engineering requirement with a measurable outcome, not as a report that gets generated after the fact. The questions we care about are whether an explanation is faithful to what the model actually computed, whether it is stable under small changes, and whether it helps a real person make a better decision.
What we work on
- Faithfulness testing. Treating explanation methods as hypotheses to be falsified, with ablation and perturbation checks built into the evaluation.
- Explanations for systems. Tracing a decision across retrieval, model calls, and post-processing so the explanation covers the whole pipeline.
- Explanations for experts. Formats designed with the domain expert who will use them, and tested for whether they change decisions.