作者位次 #6/34ACL 2026Poster· Natural Language Processing
Reinforcement Learning on Pre-Training Data
Siheng Li · Kejiao Li · Zenan Xu · Guanhua Huang · Kun Li · Haoyuan Wu · Wujiajia · Zihao Zheng · Chenchen Zhang · Kun Shi · Xue Gong · Qi Yi · Ruibin Xiong · Tingqiang Xu · Yuhao Jiang · Jianfeng Yan · Yuyuan Zeng · Guanghui Xu · Jinbao Xue · Zhijiang xu · Zheng Fang · Shuai LI · Qibin Liu · Xiaoxue Li · Zhuoyu Li · Yangyu Tao · Fei Gao · Cheng Jiang · Bochao Wang · Kai Liu · Jianchen Zhu · Wai Lam · Bo Zhou · Di Wang
摘要
Recent progress in large language models (LLMs) is largely driven by scaling training compute through either pre-training with next-token prediction (NTP) or post-training with reinforcement learning (RL). The former contributes to learning broad knowledge and skills from general data, while stru...