Yite Wang

Recent systems for optimizing prompts, programs, and ML workflows typically rely on explicit outer-loop controllers such as evolutionary search, bandits, or textual-gradient methods. We ask a fundamentally different question: how much of this search policy can be internalized by a single tool-using agent? We present ReASearch, a unified framework for reasoning-driven optimization in which the agent autonomously decides what to evaluate, how to diagnose failures, which edits to make, and when to verify or restart. Rather than serving only as a proposal generator guided by hand-designed heuristics, the agent actively analyzes outcomes, allocates budget, and refines its strategy over long horizons through persistent memory. With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML workflows. Across 14 diverse tasks, it is competitive with and mostly better than specialized optimization systems, achieving gains of 2\% to 40\% over strong domain-specific baselines, and in some cases discovering solutions that improve on prior human best-known results. Crucially, we observe that complex search behaviors, which are typically implemented by explicit controllers, emerge naturally from the agent's reasoning process.

Mid-training is increasingly recognized as a critical stage for shaping the capabilities of large language models. Recent work has shown that targeted mid-training can strengthen reasoning-intensive abilities such as math and science, and can also improve agentic capabilities in software-engineering settings. In this work, we study the parallel but less explored agentic capability: general tool use. We present MidTool, an open corpus construction pipeline for agentic tool-use mid-training that combines large-scale web, PDF, and code data with synthesized supervision from real-world tool APIs, MCP skills, and document-grounded workflows. MidTool is designed to teach models how to recognize tool affordances, ground arguments from context, compose tool call workflow, and recover from incomplete information. We mid-train Qwen3-4B-Base and Qwen3-8B-Base on MidTool-Mix, and then apply follow-up post-training with both supervised fine-tuning and reinforcement learning. Compared with baselines, MidTool-Mix consistently improves downstream performance under both SFT and RL on BFCL, $\tau^2$-Bench, and MCP Universe. These results suggest that general tool use, like other important LLM capabilities, benefits from dedicated mid-training rather than being left entirely to post-training.

Large Language Models (LLMs) increasingly rely on multi-tool orchestration to resolve complex enterprise queries. However, answering hybrid questions requires tightly coupled reasoning steps between structured data retrieval and unstructured semantic search, where a single execution error triggers an irreversible cascade. Training models to navigate these strict cross-modal pipelines via standard Reinforcement Learning (RL) is notoriously difficult. Because successful multi-hop trajectories are rare, unguided exploration leads to severe data waste and sparse rewards. While recent algorithmic advancements improve RL optimization dynamics, they still fundamentally rely on the highly inefficient process of discovering successful trajectories from scratch. In this paper, we advocate for a data-algorithm co-design paradigm to overcome this exploration bottleneck. First, we introduce HyDRA-Bench, a novel environment simulating realistic hybrid reasoning where agents must seamlessly interleave SQL and search. Second, we propose TRACE-GRPO, an adaptive RL framework that tightly integrates the data generation pipeline with policy optimization. By converting intermediate reasoning traces generated during dataset synthesis into executable, action-level hints that serve as proxy goals, HyDRA-Bench transforms sparse unguided rollouts into a dense, highly efficient curriculum. This approach not only stabilizes optimization by preventing gradient vanishing but also drives ``in-context to in-weights'' skill elicitation. Empirical results demonstrate that TRACE-GRPO drastically accelerates convergence, enabling an 8B open-weights model to surpass Claude-Opus-4.5 on hybrid reasoning, while exhibiting robust generalization to broader multi-hop text environments like MuSiQue.