作者位次 #4/7ACL 2026Poster· Natural Language Processing
SRA : Span Representation Alignment for Large Language Model Distillation
Quoc Phong Dao · Hoang-Son Nguyen · Pham Khanh Chi · Tung Nguyen · Linh Ngo Van · Nguyen Thi Ngoc Diep · Trung Le
摘要
Cross-Tokenizer Knowledge Distillation (CTKD) enables knowledge transfer between a large language model and a smaller student, even when they employ different tokenizers. While existing approaches mainly focus on token-level alignment strategies, which are often brittle and sensitive to discrepan...