作者位次 #4/8ACL 2026Poster· ACL Industry Track
Mixture of Heterogeneous Grouped Experts for Language Modeling
Zhicheng Ma · Xiang Liu · Zhaoxiang Liu · Ning Wang · Yi Shen · Kai Wang · Shuming Shi · Shiguo Lian
摘要
Large Language Models (LLMs) based on Mixture-of-Experts (MoE) are pivotal in industrial applications for their ability to scale performance efficiently. However, standard MoEs enforce uniform expert sizes, creating a rigidity that fails to align computational costs with varying token-level compl...