Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
Yuxuan Jiang · Runchao Li · Shubhashis Roy Dipta · Dawei Li · Zhao Yang
首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。
实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。
Yuxuan Jiang · Runchao Li · Shubhashis Roy Dipta · Dawei Li · Zhao Yang
Hengyuan Zhang · Shiping Yang · Xiao Liang · Chenming Shang · Yuxuan Jiang · Chaofan Tao · Jing Xiong · Hayden Kwok-Hay So · Ruobing Xie · Angel X. Chang · Ngai Wong
Training student models on synthetic data generated by strong teacher models is a promising approach to distilling the capabilities of teachers. However, existing studies reveal that stronger models are not always optimal teachers, suggesting a mismatch between the teacher’s output and the studen...
Yuxuan Jiang · Dawei Li · Francis Ferraro
While Large Reasoning Models (LRMs) excel at complex tasks via long Chain-of-Thought (CoT) reasoning, their outputs are often excessively verbose, leading to inefficiency. This problem is amplified when the student’s long-form reasoning mismatches the concise outputs of smaller teacher models—com...