作者位次 #5/8ACL 2026Poster· Natural Language Processing
H etero C ache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
Zhiyuan Shi · Qibo Qiu · Xuefeng · Zhonglin Jiang · Li Yu · Jian Jiang · Xiaofei He · Wenxiao Wang
摘要
The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods often fail to preserve globally important information. Although recent dynamic retrieval approaches attempt to address this issue, they typically su...