作者位次 #3/10ACL 2026Poster· Natural Language Processing
RRA tention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
Siran Liu · Guoxia Wang · Sa Wang · Jinle Zeng · Haoyang Xie · Siyu Lou · Jiabin Yang · Dianhai Yu · Haifeng Wang · Chao Yang
摘要
The quadratic complexity of attention mechanisms poses a critical bottleneck for large language models processing long contexts. While dynamic sparse attention methods offer input-adaptive efficiency, they face fundamental trade-offs: requiring preprocessing, lacking global evaluation, violating...