作者位次 #5/5ACL 2026Findings· Findings of ACL
SDC - L o RA : Singular-Subspace Drift Controlled L o RA to Mitigate Knowledge Forgetting
Geyuan Zhang · Xiaofei Zhou · Shihao Liu · Jingyuan Tian · Jizheng Ma
摘要
Knowledge forgetting is a central challenge when adapting LLMs to new tasks. Prior studies indicate that pretrained knowledge is concentrated in the principal singular subspace of pretrained weight W_0 ; so recent Low-Rank Adaptation (LoRA) variants initialize LoRA in the minor subspace to steer...