一作ACL 2026Findings· Findings of ACL
Astra: Activation-Space Tail-Eigenvector Low-Rank Adaptation of Large Language Models
Kainan Liu · Yong Zhang · Ning Cheng · Yun Zhu · Yanmeng Wang · Shaojun Wang · Jing Xiao
摘要
Parameter-Efficient Fine-Tuning (PEFT) methods, especially LoRA, are widely used for adapting pre-trained models to downstream tasks due to their computational and storage efficiency. However, in the context of LoRA and its variants, the potential of activation subspaces corresponding to tail eig...