作者位次 #3/5ACL 2026Poster· Natural Language Processing
SPENCE : A Syntactic Probe for Detecting Contamination in NL 2 SQL Benchmarks
Mohammadtaher Safarzadeh · Hitesh Laxmichand Patel · Afshin Oroojlooy · Graham Horwood · Dan Roth
摘要
Large language models (LLMs) have achieved strong performance on natural language to SQL (NL2SQL) benchmarks, yet their reported accuracy may be inflated by contamination from benchmark queries or structurally similar patterns seen during training. We introduce SPENCE (Syntactic Probing and Evalu...