作者位次 #10/14ACL 2026Poster· Natural Language Processing
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Xin Cheng · Wangding Zeng · Damai Dai · Qinyu Chen · Bingxuan Wang · Zhenda Xie · Kezhao Huang · Xingkai Yu · Zhewen Hao · Han Zhang · Yu-Kun Li · Huishuai Zhang · Dongyan Zhao · Wenfeng Liang
摘要
Mixture-of-Experts (MoE) scales capacity via conditional computation, but Transformers lack a native knowledge lookup primitive. We introduce conditional memory, instantiated via Deep Sparse Embedding (DSE), which indexes a massive embedding table using local n-grams for retrieval. We formalize s...