一作ACL 2026Findings· Findings of ACL
L ayer N orm Induces Recency Bias in Transformer Decoders
Junu Kim · Xiao Liu · Zhenghao Lin · Lei Ji · Yeyun Gong · Edward Choi
摘要
Causal self-attention provides positional information to Transformer decoders. Prior work has shown that stacks of causal self-attention layers alone induce a positional bias in attention scores toward earlier tokens. However, this differs from the bias toward later tokens typically observed in T...