Qihan Ren

Evaluating the safety of LLM-based agents is an increasingly important challenge, as risks in realistic deployments often emerge gradually over multi-step interactions rather than from isolated prompts or final responses. Existing trajectory-level benchmarks, however, remain limited in three key dimensions: (1) insufficient interaction diversity, due to restricted tool ecosystems and narrow scenario coverage; (2) limited observability of safety failures, as coarse labels fail to capture how risks arise and evolve; and (3) lack of long-horizon realism, where short or simplified trajectories underestimate delayed and context-dependent risks. To address these limitations, we introduce ATBench, a trajectory-level benchmark that enables structured, diverse, and realistic evaluation of agent safety. ATBench formulates agentic safety risks along three orthogonal dimensions: risk source, failure mode, and real-world harm, which provides a controllable framework for capturing diverse risk patterns. Building on this formulation, we construct trajectories using a taxonomy-guided generation engine with heterogeneous tool pools and incorporate a long-context delayed-trigger protocol that models realistic risk emergence through multi-stage interactions. The benchmark contains 1,000 trajectories (503 safe, 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools. Data quality is ensured through rule-based \& LLM-based filtering and a human full audit. Experiments across frontier LLMs, open-source models, and specialized guard systems show that ATBench remains challenging even for the strongest evaluators---GPT-5.4 achieves only 76.7\% F1 on binary safety classification and 33.6\% on fine-grained risk-source diagnosis---while enabling taxonomy-stratified analysis, cross-benchmark comparison, and diagnosis of long-horizon failure patterns.

A prevailing narrative in LLM post-training holds that supervised fine-tuning (SFT) memorizes while reinforcement learning (RL) generalizes. We revisit this claim for reasoning SFT with long chain-of-thought (CoT) supervision and find that cross-domain generalization is not absent but conditional, jointly shaped by optimization dynamics, training data, and base-model capability. Some reported failures are under-optimization artifacts: cross-domain performance first degrades before recovering and improving with extended training (a dip-and-recovery pattern), so short-training checkpoints can underestimate generalization. Data quality and structure both matter: low-quality solutions broadly hurt generalization, while verified long-CoT traces yield consistent cross-domain gains. Model capability is essential: stronger models internalize transferable procedural patterns (e.g., backtracking) even from a toy arithmetic game, while weaker ones imitate surface verbosity. This generalization is asymmetric, however: reasoning improves while safety degrades, reframing the question from whether reasoning SFT generalizes to under what conditions and at what cost.