作者位次 #8/8ACL 2026Findings· Findings of ACL
Deputy: Accelerating Large Language Model Inference with Dynamic Low-Rank Substitution
Yuhua Zhou · Shichao Weng · Changhai Zhou · Yuhan Wu · Qian Qiao · Jun Gao · Fei Yang · Aimin Pan
摘要
While the massive scale of modern LLMs enables remarkable performance, their static, input-agnostic computational graph incurs substantial resource wastage and high latency during inference. Existing dynamic schemes, such as early-exit and layer-drop reduce FLOPs but break batch processing or int...