Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?
Ziqian Zhong · Aashiq Muhamed · Mona T. Diab · Virginia Smith · Aditi Raghunathan
A new interpretability benchmark that removes elicitation confound; gradient-based methods do well.
摘要
Mechanistic interpretability is often motivated for alignment auditing, where a model's verbal explanations can be absent, incomplete, or misleading. Yet many evaluations do not control whether black-box prompting alone can recover the target behavior, so apparent gains from white-box tools may reflect elicitation rather than internal signal; we call this the *elicitation confounder*. We introduce Pando, a model-organism benchmark that breaks this confound via an *explanation axis*: models are trained to produce either faithful explanations of the true rule, no explanation, or confident but unfaithful explanations of a disjoint distractor rule. Across 720 finetuned models implementing hidden decision-tree rules, agents predict held-out model decisions from $10$ labeled query--response pairs, optionally augmented with one interpretability tool output. When explanations are faithful, black-box elicitation matches or exceeds all white-box methods; when explanations are absent or misleading, gradient-based attribution improves accuracy by 3--5 percentage points, and relevance patching, RelP, gives the largest gains, while logit lens, sparse autoencoders, and circuit tracing provide no reliable benefit. Variance decomposition suggests gradients track *decision computation*, which fields causally drive the output, whereas other readouts are dominated by *task representation*, biases toward field identity and value. We release all models, code, and evaluation infrastructure.