作者位次 #13/19ACL 2026Poster· Natural Language Processing
VRPO : Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training
Dingwei Zhu · Shihan Dou · Zhiheng Xi · Senjie Jin · Guoqiang Zhang · Jiazheng Zhang · Junjie Ye · Mingxu Chai · Enyu Zhou · Ming Zhang · Yuhui Wang · Caishuang Huang · Chenhao Huang · Yunke Zhang · Yuran Wang · Tao Gui · Qi Zhang · Xipeng Qiu · Xuanjing Huang
摘要
Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete reward supervision, which undermines policy stability and generalization. Such noise may cause models to ignore key information or even collapse in advantage estimation. We find that a strong value m...