Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin · Pengpeng Shao · Zhengqi Wen · Jinyang Wu · Mingkuan Feng · Shuo Yang · Chu Yuan Zhang · Jianhua Tao
首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。
实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。
Ruihan Jin · Pengpeng Shao · Zhengqi Wen · Jinyang Wu · Mingkuan Feng · Shuo Yang · Chu Yuan Zhang · Jianhua Tao
Jinyang Wu · Mingkuan Feng · Shuai Zhang · Feihu Che · Zhengqi Wen · Chonghua Liao · Ling Yang · Haoran Luo · Zheng Lian · Jianhua Tao
In-context learning (ICL) leverages demonstrations to enhance the performance of large language models (LLMs). However, traditional ICL struggles with complex reasoning mainly due to superficial, example-level implicit imitation. To address these limitations, we introduce ThoughtICR , an automate...
Mingkuan Feng · Jinyang Wu · Siyuan Liu · Shuai Zhang · Hongjian Fang · Ruihan Jin · Feihu Che · Pengpeng Shao · Zhengqi Wen · Jianhua Tao
The deployment of large language models (LLMs) is largely hindered by their large number of parameters. Structural pruning has emerged as a promising solution. Prior structured pruning methods directly remove unimportant parameters based on certain metrics, which often causes knowledge loss and n...
Jinyang Wu · Chonghua Liao · Mingkuan Feng · Shuai Zhang · Zhengqi Wen · Haoran Luo · Ling Yang · Huazhe Xu · Jianhua Tao
Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO often rely on unstructured self-sampling to fit scalar rewards, often producing inefficient rollouts that fail to capture transferable problem-solving strategies....
Jinyang Wu · Mingkuan Feng · Guocheng Zhai · Shuai Zhang · Zheng Lian · Fangrui Lv · Pengpeng Shao · Ruihan Jin · Zhengqi Wen · Jianhua Tao