作者位次 #4/10OSDI 2026Oral· Operating Systems & Distributed Systems
StriaTrace: Efficient Tracing and Diagnosis for Online LLM Inference (Operational Systems)
Haonan Wu · Yanqing Chen · Kun Qian · Xue Li · Jingbo Xu · Erci Xu · Ennan Zhai · Wenyuan Yu · Guangtao Xue · Jingren Zhou
摘要
Large Language Model (LLM) inference services in production operate under stringent, fine-grained Service Level Objectives (SLOs). Unlike throughput-oriented LLM training, even sporadic performance anomalies during inference can violate SLOs, underscoring the need for improved tracing and diagnos...