一作ICML 2026Poster· Optimization / Everything Else
RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
Shenyang Deng · Zhuoli Ouyang · Tianyu Pang · Zihang Liu · Ruochen Jin · Shuhua Yu · Yaoqing Yang
首次获取该候选人的完整档案及 StartLux 评估共 20 积分；每位用户单独解锁，之后查看免费。主动刷新 20 积分。
实时查找电子邮箱，费用不超过20积分；未找到邮箱不扣费。
Shenyang Deng · Zhuoli Ouyang · Tianyu Pang · Zihang Liu · Ruochen Jin · Shuhua Yu · Yaoqing Yang
Tianyu Pang · Vignesh Kothapalli · Shenyang Deng · Haohui Wang · Dawei Zhou · Yaoqing Yang
Tianyu Pang · Yujie Fang · Zihang Liu · Shenyang Deng · Lei Hsiung · Shuhua Yu · Yaoqing Yang
Muon has recently shown promising results in LLM training. In this work, we study how to further improve Muon. We argue that Muon’s orthogonalized update rule suppresses the emergence of heavy-tailed weight spectra and over-emphasizes the training along noise-dominated directions. Motivated by th...