LLMs Encode Their Failures: Predicting Success from Pre-Generation Activations
William Lugoloobi · Thomas Foster · William Bankes · Chris Russell
LLMs encode their likelihood of success in pre-generation activations. Linear probes extract this signal across reasoning modes to predict success and enable cost-aware routing, matching high-compute accuracy at much lower inference cost.
摘要
Running LLMs with extended reasoning on every problem is expensive, but determining which inputs actually require additional compute remains challenging. We investigate whether a model's likelihood of success is recoverable from its internal representations prior to generation, and whether this signal can guide more efficient inference. We train linear probes on pre-generation activations to predict policy-specific success on math and coding tasks, substantially outperforming surface features such as question length and TF-IDF. Using E2H-AMC, which provides both human and model performance on identical problems, we show that models encode a model-specific notion of difficulty that is distinct from human difficulty, and that this distinction increases with extended reasoning. Leveraging these probes, we demonstrate that routing queries across a pool of models can match the performance of the best-performing model whilst reducing inference cost by up to 70\% on MATH, showing that internal representations enable practical efficiency gains even when they diverge from human intuitions about difficulty.