Zhijing Jin

Large Language Models (LLMs) are trained to refuse harmful requests, yet they remain vulnerable to jailbreak attacks that exploit weaknesses in conversational safety mechanisms. We introduce Incremental Completion Decomposition ICD, a trajectory-based jailbreak strategy that elicits a sequence of single-word continuations related to a malicious request before eliciting the full response. In addition, we propose ICD variants that use model-generated or attacker-injected intermediate continuations, as well as final-response prefilling. We evaluate these variants across a broad set of open-weight model families, demonstrating superior Attack Success Rate (ASR) on AdvBench, JailbreakBench, and StrongREJECT compared to existing methods. In addition, we provide a theoretical account of why ICD is effective and present mechanistic evidence that successful attack trajectories suppress refusal-related representations and shift activations away from safety-aligned states.

The rise of autonomous AI agents suggests that dynamic benchmark envi- ronments with built-in feedback on scientifically grounded tasks are needed to evaluate the capabilities of these agents in research work. We introduce STARGAZER, a scalable environment for evaluating AI agents on dynamic, iterative physics-grounded model-fitting tasks using inference on radial- velocity (RV) time series data. STARGAZER comprises 120 tasks across three difficulty tiers, including 20 real archival cases, covering diverse scenarios ranging from high-SNR single-planet systems to complex multi-planetary configurations requiring involved low-SNR analysis. Our evaluation of eight frontier agents reveals a gap between numerical optimization and adherence to physical constraints: although agents often achieve a good statistical fit, they frequently fail to recover correct physical system pa- rameters, a limitation that persists even when agents are equipped with domain-expert skills. Furthermore, increasing test-time compute yields only marginal gains, with excessive token usage often reflecting recursive failure loops rather than meaningful exploration. STARGAZER presents an opportunity to train, evaluate, scaffold, and scale strategies on a model- fitting problem of practical research relevance today. Our simulation-driven methodology may also provide a useful design pattern for other scientific model-fitting domains.

Large language model (LLM) agents act on inputs users do not control, including tool returns, retrieved documents, and messages. Indirect prompt injection embeds instructions in these channels to redirect an agent toward unintended actions. The same tool call may be appropriate for one task and harmful for another, making this threat difficult to address with input-level defenses alone. We ask whether robustness against indirect prompt injection can be built into the LLM itself, and instantiate this idea with Odile, a low-rank adapter trained on paired benign and harmful traces generated by the base model. Odile is designed to convert attacker-directed trajectories into a no-action state before a valid malicious tool call is produced, while preserving ordinary behavior on benign inputs. In matched evaluations, this single model-level defense matches or improves upon the security–utility tradeoffs of prompt hardening, runtime detection and rewriting, and prior fine-tuning defenses. Across three prompt-injection benchmarks, eight model backbones, and two defense-aware adaptive settings, Odile consistently reduces attack success while retaining benign behavior and ordinary model capability. It transfers from a compact training dataset to longer trajectories, different tool interfaces, mixture-of-experts models, and hybrid attention architectures. We release Odile at https://github.com/memo-ozdincer/odile