作者位次 #14/15ACL 2026Poster· Natural Language Processing
O cto B ench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding
Deming Ding · Shichun Liu · Enhui Yang · Jiahang Lin · Ziying Chen · Shihan Dou · Honglin Guo · Weiyu Cheng · Pengyu Zhao · Chengjun Xiao · Qunhong Zeng · Qi Zhang · Xuanjing Huang · Qidi Xu · Tao Gui
摘要
Modern coding scaffolds turn LLMs into capable software agents, but their ability to follow scaffold-specified instructions remains under-examined, especially when constraints are heterogeneous and persist across interactions. To fill this gap, we introduce OctoBench, which benchmarks scaffold-aw...