一作ACL 2026Poster· Natural Language Processing
Distillation Traps and Guards: A Calibration Knob for LLM Distillability
Weixiao Zhan · Yongcheng Jing · Leszek Rutkowski · Dacheng Tao
摘要
Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks. Our analysis revealed several distillation traps: tail noise, off-policy instability, and, most fundamentally, the teache...