作者位次 #6/12ACL 2026Findings· Findings of ACL
WISCA : A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
Jiacheng Li · Jianchao Tan · Zhidong Yang · Pingwei Sun · FEIYE HUO · Jiayu Qin · Xiangyu Zhang · Maoxin He · Guangming Tan · Weile Jia · Xunliang Cai · Tong Zhao
摘要
Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architectural modifications or optimizer adjustments. However, these approaches lack systematic optimization of weight pattern...