作者位次 #3/12ACL 2026Findings· Findings of ACL
Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping
Yao Chen · Yilong Chen · Yinqi Yang · Junyuan Shang · Zhenyu Zhang · Zefeng Zhang · Shuaiyi Nie · Shuohuan Wang · YU SUN · hua wu · Haifeng Wang · Tingwen Liu
摘要
Existing approaches to increasing the effective depth of Transformers predominantly rely on parameter reuse, extending computation through recursive execution.Under this paradigm, the network structure remains static along the training timeline, and additional computational depth is uniformly ass...