作者位次 #3/7ACL 2026Poster· Natural Language Processing
V ec I nfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
Dingyu Yao · Chenxu Yang · Zhengyang Tong · Zheng Lin · WEI LIU · Jian Luan · Weiping Wang
摘要
The Key-Value (KV) cache introduces substantial memory overhead during large language model (LLM) inference. Although existing vector quantization (VQ) methods reduce KV cache usage and provide flexible representational capacity across bit-widths, they suffer severe performance degradation at ult...