作者位次 #20/21OSDI 2026Oral· Operating Systems & Distributed Systems
Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning
Shan Yu · Yifan Qiao · Mingyuan Ma · Yangmin Li · Shuo Yang · Xinyuan Tong · Yang Wang · Zhiqiang Xie · Yuwei An · Shiyi Cao · Ke Bao · Deepak Vij · Xiaoning Ding · Yichen Wang · Qingda Lu · Zhong Wang · Gao Gao · Harry Xu · Junyi Shu · Jiarong Xing · Ying Sheng
摘要
Inference providers must maintain availability for many LLMs, including low-volume but essential models, making resource efficiency increasingly important as token prices fall. Analysis of production traces reveals a dynamic _bursty-group pattern_ in which sets of models become active together an...