作者位次 #5/8OSDI 2026Oral· Operating Systems & Distributed Systems
Efficient LLM Serving on Commodity GPU Clusters with Data-Reduced Cross-Instance Orchestration
Jiangsu Du · Hongbin Zhang · Taosheng Wei · Zhenyi Zheng · Jiazhi Jiang · Kaiyi Wu · Zhiguang Chen · Yutong Lu
摘要
Existing LLM serving strategies can be categorized by whether prefill and decode phases are disaggregated: non-disaggregated (NoDG) or fully disaggregated (FuDG). However, they neither fit commodity GPU clusters, which remain widely deployed as mainstream AI infrastructure. NoDG suffers from seve...