作者位次 #5/10ACL 2026Findings· Findings of ACL
Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learning
Zhenpeng Su · Leiyu Pan · Minxuan Lv · Tiehua Mei · Zijia Lin · Yuntao Li · Wenping Hu · Ruiming Tang (Huawei Noah' · Kun Gai · Guorui Zhou
摘要
Large language model post-training relies on reinforcement learning to improve model capability and alignment quality. However, the off-policy training paradigm introduces distribution shift, which often pushes the policy beyond the trust region, leading to training instabilities manifested as fl...