作者位次 #3/5ACL 2026Findings· Findings of ACL
How to Set the Learning Rate for Large-Scale Pre-training?
Yunhua Zhou · Shuhao Xing · Junhao Huang · Xipeng Qiu · Qipeng Guo
摘要
Optimal configuration of the learning rate (LR) is a fundamental yet formidable challenge in large-scale pre-training. Given the stringent trade-off between training costs and model performance, the pivotal question is whether the optimal LR can be accurately extrapolated from low-cost experiment...