一作ACL 2026Findings· Findings of ACL
Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization
Huimin Xu · Shuai Zhao · Xiaobao Wu · Anh Tuan Luu
摘要
Reinforcement learning with verifiable rewards (RLVR) has become an effective paradigm for improving the reasoning ability of large language models. However, widely used RLVR algorithms, such as GRPO, often suffer from entropy collapse, leading to premature determinism and unstable optimization....