paper.dinq.meAI 与前沿科学顶会 · 学术新星雷达
登录 DINQ
返回新星榜
NL

Nancy Lau

COLM 2026评测与基准代码智能体与工具
新星分贡献者
39
一作贡献0
核心/参与12
方向广度6
合作网络9
0 一作 · 1 篇 · COLM 2026

获取完整研究档案

首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。

联系方式

实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。

2026 顶会论文

1
作者位次 #2/11COLM 2026Poster· Poster Session 5· Imperial Ballroom· #41

Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows

Hardy Chen · Nancy Lau · Haoqin Tu · Shuo Yan · Xiangyan Liu · Zijun Wang · Juncheng Wu · Michael Qizhe Shieh · Alvaro A. Cardenas · Cihang Xie · Yuyin Zhou

Frontier coding agents exhibit exploitation behavior under multi-turn user pressure when tasked at developing better ML models.

摘要

Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score, namely the reported score on a public evaluation file in the workspace, rather than through direct inspection of the agent's intermediate outputs. We study whether multi-round user pressure to improve that score induces public-score exploitation: behavior that raises the public score through shortcuts without improving hidden private evaluation. We begin with a preliminary single-script tabular classification task, where GPT-5.4 and Claude Opus 4.6 exploit in all 10 runs. We then build \name, a 34-task machine-learning repository benchmark spanning three input modalities, and collected 1314 multi-round trajectories from 13 frontier coding agents. Across \name, we observe 498 exploitative runs, spanning across all tasks. We also find that stronger models have higher exploitation rates, supported by a significant Spearman rank correlation of 0.82. Our ablation experiments show that higher user pressure leads to earlier exploitation, manifested itself in the average first exploit round reducing by 6.8 rounds (\textit{i.e.}, 11.42 to 4.58). As a mitigation, explicit anti-exploit wordings in prompt completely eliminates exploitation while keeping public labels exposed. These results show that repeated pressure to improve a public score can induce public-score exploitation in coding-agent workflows, and that prompt design can materially change that behavior.

评测与基准代码智能体与工具coding agentvibe researchmachine learninglarge language modelsOpenReview

常见合作者