一作OSDI 2026Oral· Operating Systems & Distributed Systems
Inference in the Shadows: Taming Memory Bandwidth Contention in Mobile LLM Inference with Sereno
Tong Xin · Xinrui Shi · Mingkai Dong · Zeyu Mi
摘要
The proliferation of large language models (LLMs) on mobile devices introduces a new performance challenge: resource contention between compute-intensive inference and latency-sensitive foreground applications. We identify a severe and _asymmetric_ interference where concurrent LLM inference subs...