一作ACL 2026Findings· Findings of ACL
H qe KV : Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference
He Wang · Yu Gu · Fangfang Li · Zhigang Wang · Zhenghao Liu · Ning Wang · Xiaohua Li · Ge Yu
摘要
The autoregressive inference in large language models requires repeated computation across transformer layers. While caching intermediate key-value (KV) pairs eliminates redundancy, it introduces severe memory overhead, particularly in long-context settings. Most existing cache compression method...