作者位次 #5/7ACL 2026Poster· Natural Language Processing
T alk L o RA : Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
Lin Mu · Haiyang Wang · Li Ni · Lei Sang · Zhize Wu · Peiquan Jin · Yiwen Zhang
摘要
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of Large Language Models (LLMs), and recent Mixture-of-Experts (MoE) extensions further enhance flexibility by dynamically combining multiple LoRA experts. However, existing MoE-augmented LoRA methods assume that experts operate i...