Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
Tianyu Pang · Vignesh Kothapalli · Shenyang Deng · Haohui Wang · Dawei Zhou · Yaoqing Yang
首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。
实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。
Tianyu Pang · Vignesh Kothapalli · Shenyang Deng · Haohui Wang · Dawei Zhou · Yaoqing Yang
Vignesh Kothapalli · Rishabh Ranjan · Valter Hudovernik · Vijay Prakash Dwivedi · Johannes Hoffart · Carlos Guestrin · Jure Leskovec
Vignesh Kothapalli · Ata Fatahibaarzi · Hamed Firooz · Maziar Sanjabi
Chain-of-thought (CoT) prompting combined with few-shot in-context learning (ICL) has unlocked significant reasoning capabilities in large language models (LLMs). However, ICL with CoT examples is ineffective on novel tasks when the pre-training knowledge is insufficient. We study this problem in...
Wei-Rui Chen · Vignesh Kothapalli · Ata Fatahibaarzi · Hejian Sang · shao tang · Qingquan Song · Zhipeng Wang · Muhammad Abdul-Mageed
Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge distillation (KD) over lengthy sequences with prompt (P), chain-of-thought (CoT), and answer (A) sections makes the proces...