C ore C ode B ench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks
Lingyue Fu · Hao Guan · Bolun Zhang · Haowei Yuan · Yaoming Zhu · Lin Qiu · ZongYu Wang · Xuezhi Cao · Xunliang Cai · Weiwen Liu · Ruiming Tang (Huawei Noah' · s Ark Lab)
摘要
The evaluation of Large Language Models (LLMs) for software engineering has shifted towards complex, repository-level tasks. However, existing benchmarks predominantly rely on coarse-grained pass rates that treat programming proficiency as a monolithic capability, obscuring specific cognitive bot...