Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?
H S V N S Kowndinya Renduchintala · Sumit Bhatia
摘要
Large Language Models (LLMs) exhibit a puzzling disparity in their formal linguistic competence: while they learn some linguistic phenomena with near-perfect mastery, they often perform below chance on others, even after training on trillions of tokens. In this work, we investigate whether these...