作者位次 #3/6ACL 2026Poster· Natural Language Processing
MTA : Multi-Granular Trajectory Alignment for Large Language Model Distillation
Pham Khanh Chi · Quoc Phong Dao · Thuat Nguyen · Linh Ngo Van · Trung Le · Thanh Hong Nguyen
摘要
Knowledge distillation is a key technique for compressing large language models (LLMs), but most existing methods align representations at fixed layers or token-level outputs, ignoring how representations evolve across depth. As a result, the student is only weakly guided to capture the teacher’s...