Spectral Disentanglement: Rank-Aware Task Adaptation for Rehearsal-free Continual Learning in LLM s
Huanxuan Liao · Shizhu He · Yupu Hao · Yequan Wang · Wenhao Teng · Xiangwen Liao · Jun Zhao · Kang Liu
摘要
Continual Learning (CL) for Large Language Models (LLMs) faces a fundamental Stability-Plasticity Dilemma: balancing the plasticity to acquire new capabilities with the stability to preserve prior knowledge. While Parameter-Efficient Fine-Tuning methods, such as LoRA, enable efficient adaptation,...