作者位次 #5/5ACL 2026Findings· Findings of ACL
RLHS : Mitigating Misalignment in RLHF with Hindsight Simulation
Kaiqu Liang · Haimin Hu · Ryan Liu · Thomas L. Griffiths · Jaime Fernández Fisac
摘要
While Reinforcement Learning from Human Feedback (RLHF) has shown promise in aligning generative AI, we present empirical evidence that it can also cause severe, systematic misalignment. We hypothesize that this stems from evaluator feedback depending on downstream outcome predictions (foresight)...