Verbalizing LLMs' assumptions to explain and control sycophancy
Myra Cheng · Isabel Sieh · Humishka Zope · Sunny Yu · Lujain Ibrahim · Aryaman Arora · Jared Moore · Desmond Ong · Dan Jurafsky · Diyi Yang
We provide evidence of causal links between models' internal assumptions and downstream sycophancy, and surface these assumptions using Verbalized Assumptions; we show that assumption probes can steer LLM sycophancy.
摘要
LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment. We hypothesize that this behavior arises from LLMs' incorrect assumptions about the user, like underestimating how often users are seeking information over reassurance. We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs. Verbalized Assumptions provide insight into LLM sycophancy, delusion, and other safety issues: in social sycophancy datasets, "seeking validation" is the most frequent bigram in LLMs' assumptions. We provide evidence for a causal link between assumptions and sycophantic model behavior: we train linear probes on internal representations associated with Verbalized Assumptions and then use these probes for interpretable, fine-grained steering of social sycophancy. Finally, we identify a *human-AI expectation gap* that explains why LLMs default to sycophantic assumptions. On identical queries, people expect more objective and informative responses from AI than from other humans, but LLMs trained on human-human conversation do not account for this difference in expectations. Our work contributes a new understanding of assumptions as a mechanism for analyzing and controlling sycophancy.