paper.dinq.meAI 与前沿科学顶会 · 学术新星雷达
登录 DINQ
返回新星榜
YF

Yicheng Fu

COLM 2026评测与基准社会与人本智能体与工具
新星分贡献者
22
一作贡献0
核心/参与5
方向广度6
合作网络3
0 一作 · 1 篇 · COLM 2026

获取完整研究档案

首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。

联系方式

实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。

2026 顶会论文

1
作者位次 #8/11COLM 2026Poster· Poster Session 6· Imperial Ballroom· #32

CooperBench: Why coding agents cannot be your teammates yet

Arpandeep Khatua · Hao Zhu · Peter Tran · Arya Prabhudesai · Frederic Sadrieh · Johann Kaspar Lieberwirth · Xinkai Yu · Yicheng Fu · Michael J Ryan · Jiaxin Pei · Diyi Yang

We introduce CooperBench, a benchmark for collaborative coding, and show that state-of-the-art coding agents perform substantially worse when required to coordinate, revealing coordination rather than coding skill as the primary bottleneck.

摘要

Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. Similarly, as AI agents increasingly collaborate on complex work, they must develop coordination capabilities to function as effective teammates. We hypothesize that current agents lack these capabilities. To test it, we introduce CooperBench, a benchmark of 652 collaborative coding tasks spanning 12 libraries and 4 languages. Each task assigns agents independently implementable features that may conflict without coordination. Tasks are grounded in real open-source repositories with expert-written tests, which makes the cooperation outcomes fully verifiable. Evaluating SOTA coding agents, we observe the curse of coordination: at least 30\% average drop in success when agents work together versus a single agent completing both tasks, especially when the tasks are not extremely easy or hard. Critically, even on task pairs where each agent can individually solve its assigned feature, the best models still fail to coordinate roughly half the time, a conditional failure rate that cannot be explained by coding difficulty alone. We also find that human developer pairs solve 9 out of 10 tasks under the same coordination constraints that agents handle at only 3 out of 10, establishing that the overhead of splitting work is not the root cause. We identify three failure modes: (1) communication channels become jammed with vague, ill-timed, and inaccurate messages; (2) even with good communication, agents deviate from their commitments and hold incorrect expectations about others; and (3) agents have troubles using version control tools to perform real-time coordination. Beyond an open-source benchmark, this work contributes a novel understanding of what agents need to learn to become effective teammates.

评测与基准社会与人本智能体与工具Social IntelligenceConflict ResolutionCooperationCoordinationOpenReview 项目页

常见合作者