作者位次 #5/9ACL 2026Findings· Findings of ACL
PBEB ench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics
Atharva Naik · Darsh Agrawal · Prakam · Yash Mathur · Manav Kapadnis · Yuwei An · Clayton Marr · Carolyn Rose · David R. Mortensen
摘要
While many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs), few isolate reasoning as a capability independent of domain knowledge. We introduce a new benchmark for inductive reasoning inspired by Sound Law Induction (SLI) in historical linguistics and formulated in a s...