作者位次 #6/10ACL 2026Poster· Natural Language Processing
S pec C ache: Speculative KV Cache Reuse for Efficient RAG Serving
Zijian Wen · Tao Zhang · Shuangwu Chen · Shenghao Ye · Yu Guo · Qirui Chen · Jingxian Shuai · Yunpeng Hou · Huasen He · Jianyang
摘要
Retrieval-Augmented Generation (RAG) significantly enhances LLMs but faces high prefill latency during long-context processing. While KV cache reuse can mitigate this, current methods relying on shallow features or static heuristics often fail to identify critical tokens for recomputation, result...