When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retraction
Yuqing Yang · Robin Jia
We find that an LLM's internal belief causally controls its retraction behaviors, a finding that holds even under supervised fine-tuning.
摘要
We study the internal mechanisms that govern when LLMs choose to retract wrong answers, i.e., spontaneously and immediately acknowledge errors in their previously generated false assertions. Using model-specific testbeds, we find that while LLMs are capable of retraction, they do so only rarely, even when they can recognize their mistakes when asked in a separate interaction. We identify a reliable predictor of retraction: the model's \emph{momentary belief}, as measured by a linear probe on its internal representation. The probe is trained to predict the correctness of answers on external datasets unrelated to retraction, then applied to settings where models should retract. A model retracts only when it ``believes'' its answers to be incorrect \emph{during generation}; these beliefs frequently diverge from models' parametric knowledge as measured by factoid questions. Steering experiments further demonstrate that model belief causally drives retraction. In particular, when the model believes its answer to be incorrect, this not only encourages the model to attempt further verification, but also alters attention dynamics to promote retraction. Finally, we show that supervised fine-tuning re-uses this existing mechanism linking belief with retraction, and primarily improves retraction performance by helping the model learn more accurate internal beliefs.