SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
Tiancheng Hu · Joachim Baumann · Lorenzo Lupo · Nigel Collier · Dirk Hovy · Paul Röttger
首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。
实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。
Tiancheng Hu · Joachim Baumann · Lorenzo Lupo · Nigel Collier · Dirk Hovy · Paul Röttger
Yijiang River Dong · Tiancheng Hu · Zheng Hui · Caiqi Zhang · Ivan Vulić · Andreea Bobu · Nigel Collier
Large Language Model (LLM) agents deployed for real-world tasks face a fundamental dilemma: user requests are underspecified, yet agents must decide whether to act on incomplete information or interrupt users for clarification. Existing approaches either rely on brittle confidence thresholds that...
Yixuan Jiang · Tiancheng Hu · Jose Hernandez-Orallo · David Stillwell · Luning Sun
Understanding how discussion dynamics shape team creativity has been limited by the difficulty of measuring process at scale. We introduce Trace, a corpus of 309 group discussions from 103 teams (460 participants) across six creative problem-solving tasks. The dataset follows an input-process-out...
Beiduo Chen · Tiancheng Hu · Caiqi Zhang · Robert Litschko · Anna Korhonen · Barbara Plank
Reasoning-tuned LLMs utilizing long Chain-of-Thought (CoT) excel at single-answer tasks, yet their ability to model Human Label Variation—which requires capturing probabilistic ambiguity rather than resolving it—remains underexplored. We investigate this through systematic disentanglement experim...
Caiqi Zhang · Ruihan Yang · Xiaochen Zhu · Chengzu Li · Tiancheng Hu · Yijiang River Dong · Deqing Yang · Nigel Collier
While confidence estimation is a promising direction for mitigating hallucinations in Large Language Models (LLMs), current research overwhelmingly focuses on single-turn settings. The dynamics of model confidence in multi-turn conversations, where context accumulates and ambiguity is progressive...
Tiancheng Hu · Benjamin Minixhofer · Nigel Collier
The “alignment tax” of post-training is typically framed as a drop in task accuracy. We show it also involves a severe loss of calibration, making models overconfident, less reliable, and model outputs less diverse. We demonstrate that this trade-off can be navigated effectively via a simple post...