LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent
Wanli Li · Bince Qu · Bo Pan · Jianyu Zhang · Zheng Liu · Pan Zhang · Wei Chen · Bo Zhang
We show that a virtual-world framework enabling truly scalable agentic RL is all it takes to train a 4B-parameter deep research agent that rivals Claude-4.5-Sonnet on GAIA (71.3%) and Xbench (78%).
摘要
Reinforcement Learning (RL) has emerged as a powerful training paradigm for LLM-based agents. However, scaling agentic RL for deep research remains constrained by two coupled challenges: hand-crafted synthetic data fail to elicit genuine real-world search capabilities, and real-world search dependency during RL training introduces instability and expensive cost, which limit the scalability of Agentic RL. LiteResearcher is a training framework to make Agentic RL scalable: by constructing a lite virtual world that mirrors the real-world search dynamics, we enabled a continuously improving training recipe that empowers tiny search agent to outperform large-scale open-source and commercial models (e.g. Tongyi DeepResearch and Claude-4.5 Sonnet). Specifically, on most common benchmarks like GAIA and Xbench, our LiteResearcher-4B achieves the open-source state-of-the-art results of 71.3% and 78.0% respectively, proving that scalable RL training is essential for Deep Research Agents.