作者位次 #2/6ACL 2026Findings· Findings of ACL
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLM s
Piotr Nawrot · Jianing Li · Renjie Huang · Sebastian Ruder · Kelly Marchisio · Edoardo Ponti
摘要
Sparse attention offers a promising strategy to extend long-context capabilities in Transformer LLMs, yet its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation. We address this gap with the largest-scale empirical analysis to date of training-free sparse at...