Why Do More Experts Fail? A Theoretical Analysis of Model Merging
Zijing Wang · Xingle Xu · Yongkang Liu · Yiqun Zhang · Peiqin Lin · Shi Feng · Daling Wang · Xiaocui Yang · Hinrich Schuetze
摘要
Model merging dramatically reduces storage and computational resources by combining multiple expert models into a single multi-task model. However, existing methods struggle to maintain performance gains as the number of merged models increases. In this paper, we investigate the key obstacles tha...