作者位次 #2/4ACL 2026Findings· Findings of ACL
QUARTZ : Quantile-Aware Routing and Queueing for TTFT SLO s in LLM Serving
Zhipeng Liu · Yifan Zheng · Fanqi Kong · Ziming Zhao
摘要
Large Language Model (LLM) serving systems increasingly face strict time-to-first-token (TTFT) service-level objectives (SLOs), yet TTFT remains highly sensitive to router-side queueing effects. Prefill costs scale with prompt length, decode lengths are uncertain, and prefix locality creates stro...