ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
Shangqian Gao · Ting Hua · Reza Shirkavand · Chi-Heng Lin · Zheng Tang · Zhengao Li · Longge Yuan · Fangyi Li · Zeyu Zhang · Alireza Ganjdanesh · Qian Lou · Jie Xu · Yen-Chang Hsu