作者位次 #4/6FAST 2026Oral· Storage & File Systems
Bidaw: Enhancing Key-Value Caching for Interactive LLM Serving via Bidirectional Computation–Storage Awareness
Shipeng Hu · Guangyan Zhang · Yuqi Zhou · Yaya Wei · Ziyan Zhong · Jike Chen
摘要
In interactive LLM serving, historical key–value tensors (KVs) of multi-round conversations are often cached in a two-tier storage system consisting of host memory and SSDs, which provides large capacity at low cost. However, loading KVs from two-tier storage in existing approaches increases serv...