一作ACL 2026Findings· Findings of ACL
R e F ree KV : Towards Threshold-Free KV Cache Compression
Xuanfan Ni · Liyan Xu · Chenyang Lyu · Longyue Wang · Mo Yu · Lemao Liu · Fandong Meng · Jie Zhou · Piji Li
摘要
To reduce memory consumption during LLM inference, a handful of methods have been proposed for KV cache pruning.While these techniques can accomplish lossless memory reduction on many datasets, they often hinge on an under-emphasized condition: an input/domain-specific threshold for KV cache budg...