一作ACL 2026OralBest Paper· Natural Language Processing
Characterizing the Expressivity of Local Attention in Transformers
Jiaoda Li · Ryan Cotterell
摘要
The transformer is the most popular neural architecture for language modeling. The cornerstone of the transformer is its global attention mechanism, which lets the model aggregate information from all preceding tokens before generating the next token. One common variant of attention is called loc...