作者位次 #3/4ACL 2026Poster· Natural Language Processing
R o BSA : R o PE -based Blockwise Sparse Multi-head Latent Attention
Xinyu Shi · Kairong Luo · Zhen Zheng · Wenguang Chen
摘要
Large Language Models (LLMs) have rapidly advanced in recent years, scaling up in both parameter count and context length. However, as context windows extend from thousands to hundreds of thousands of tokens, attention computation becomes the dominant source of memory usage and runtime in decodin...