Zhe Zhao

Pluralistic alignment has emerged as a critical frontier in the development of Large Language Models (LLMs), with reward models (RMs) serving as a central mechanism for capturing diverse human values. While benchmarks for general response quality are prevalent, evaluating how well reward models account for individual user preferences remains an open challenge. To bridge this gap, we introduce **Personalized RewardBench**, a novel benchmark designed to rigorously assess reward models' capacity to model personalized preferences. We construct chosen and rejected response pairs based on strict adherence to (or violation of) user-specific rubrics, ensuring that preference distinctions are uniquely tailored to the individual. In particular, human evaluations confirm that the primary discriminative factor between pairs is strictly personal preference, with both responses maintaining high general quality (e.g., correctness and helpfulness). Extensive testing reveals that existing state-of-the-art reward models struggle significantly with personalization, peaking at an accuracy of just 75.94\%. Crucially, because an effective reward model benchmark should predict a reward model's performance on downstream tasks, we conduct experiments demonstrating that our benchmark exhibits a significantly higher correlation with downstream performance under both Best-of-N (BoN) sampling and Proximal Policy Optimization (PPO) compared to existing baselines. These findings establish Personalized RewardBench as a robust and accurate proxy for evaluating reward models' performance in downstream applications.

Large Language Model (LLM) agents are increasingly used for black-box mobile application testing, where the ultimate goal is the comprehensive coverage of the features users rely on. However, most existing studies focus on Activity Coverage, which measures the proportion of distinct application screens visited during a test session but does not directly measure whether agents actually test the features users care about. To address this, we focus on two research questions: (1) how to automatically measure whether a testing framework covers the features users care about, and (2) how to design a framework that achieves more comprehensive feature coverage. To answer (1), we introduce the User-Centric Feature Coverage benchmark, which consists of two components: a manually crafted dataset of 3,012 user-centric target features based on real reviews and app descriptions, and an automated evaluation pipeline that translates agent-generated test cases and target features into a shared semantic space, then uses an LLM-as-a-judge to compute coverage. To answer (2), we introduce ECS (**E**xploration, **C**ritical User Journeys, and **S**ynthetic Reviews). ECS is a framework that explores the application, then uses an LLM to convert exploration traces into Critical User Journeys (CUJs) to bring the semantic meaning of the trace closer to a distinct feature. Each CUJ consists of an LLM-inferred user intent paired with reproducible test steps. ECS then aggregates multiple CUJs using persona-augmented synthetic reviews to capture more complex features, simulating real users who build knowledge of an application across multiple screens before writing a review. ECS achieves 64.1\% feature coverage, a 23.8\% gain over the state-of-the-art baseline, and covers 72.5\% of features in app descriptions, capturing the most prominent features. Our failure analysis shows statelessness and exploration discoverability as main reasons of failures, providing future directions.