一作ACL 2026Findings· Findings of ACL
Faster M o E LLM Inference for Extremely Large Models
Haoqi Yang · Luohe Shi · Qiwei Li · Zuchao Li · Ping Wang · Hao Huang · Hai Zhao
摘要
In fine-grained sparse Mixture-of-Experts (MoE) models, a large pool of specialized experts replaces a small homogeneous set, shifting performance and throughput to be governed by inference-time expert activation. Yet most existing optimization recipes implicitly assume a fixed activation budget...