Shenzhi Wang

Generative Reward Models (GenRMs) and LLM-as-a-Judge exhibit deceptive alignment by producing correct judgments for incorrect reasons, as they are trained and evaluated to prioritize Outcome Accuracy , which undermines their ability to generalize during RLHF. We introduce Rationale Consistency ,...