作者位次 #3/6ACL 2026Poster· Natural Language Processing
Every Token Counts: Generalizing 16 M Ultra-Long Context in Large Language Models
Xiang Hu · Zhanchao Zhou · Ruiqi Liang · Zehuan Li · Wei Wu · Jianguo Li
摘要
This work explores efficient ultra-long context modeling. We posit that an effective solution requires three fundamental properties: sparsity, random-access flexibility, and length generalization. To achieve this, we leverage Hierarchical Sparse Attention (HSA), a novel attention mechanism that s...