作者位次 #3/6ACL 2026Findings· Findings of ACL
Attention Sinks in Diffusion Language Models
Maximo Eduardo Rulli · Simone Petruzzi · Edoardo Michielon · Fabrizio Silvestri · Simone Scardapane · Alessio Devoto
摘要
Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirectional attention, enabling parallel token generation while maintaining competitive performance. Although their efficie...