作者位次 #3/8ACL 2026Findings· Findings of ACL
GRASS : Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Kaiyuan Tian · Linbo Qiao · Yu Tang · Gongqingjian Jiang · Baihui Liu · Yifu Gao · Xialin Su · Dongsheng Li
摘要
Full-parameter fine-tuning of large language models is constrained by substantial GPU memory demands. Low-rank adaptation methods mitigate this challenge by updating only a subset of parameters. However, these approaches often limit model expressiveness and yield lower performance than full-param...