一作ACL 2026Findings· Findings of ACL
Prior Beliefs Prejudice LLM -as-Judge: Evidence from Persuasion Evaluation
Pardis Sadat Zahraei · Xiaoning Wang · Nimet Beyza Bozdag · Gokhan Tur · Dilek Hakkani-Tür
摘要
Large Language Models (LLMs) are increasingly used as judges to evaluate text quality, moderate content, and assess arguments. We investigate whether alignment-instilled prior beliefs bias LLM judgments, using persuasion evaluation as a representative task. We find a systematic failure: models co...