Implicit Representations of Grammaticality in Language Models
Yingshan Susan Wang · Linlu Qiu · Zhaofeng Wu · Roger P. Levy · Yoon Kim
摘要
Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitted to maximize corpus likelihood, generate grammatically well-formed text and discriminate well between grammatical and ungrammatical sentences in...