Tian Lan

Autoregressive decoding in Transformer-based language models relies on the KV cache, whose memory footprint grows linearly with sequence length and becomes the primary bottleneck for long-context inference. KV cache eviction addresses this by retaining a fixed-size subset of key-value pairs and discarding the rest. We identify that a primary source of output degradation is not the residual attention mass on evicted tokens, which existing methods already minimize, but a directional mismatch between the retained and evicted token sets. Specifically, the evicted tokens in practice are often near-orthogonal to the retained ones. Thus, even a small evicted mass could have an oversized impact on the resulting direction distribution and amplify into substantial output error. This reveals a fundamental limit in existing strategies. To address this, we propose MomentKV, which maintains compact, small-size moment statistics over the evicted token set, including a count, key mean, value mean, and value-key covariance. During eviction, the moment statistics is leveraged to identify tokens already well aligned with and captured by the accumulated summary, keeping the evicted set geometrically regular. During inference, they yield a closed-form first-order approximation of the evicted attention output, forming a mutually reinforcing loop between selective eviction and accurate correction. On LongBench and RULER with LLaMA-3.1-8B-Instruct and Qwen3-4B-Instruct, MomentKV outperforms all baselines at every cache budget, with the largest gains under aggressive compression.

Discovering solutions for complex computer-use tasks remains a fundamental challenge due to vast action spaces, non-reversible state transitions, and sparse rewards. Existing approaches for Graphical User Interface (GUI) agents operate as unidirectional processes, where the lack of regressive ability prevents the reuse of partial successes and recovery from early missteps. We introduce Agent Alpha, a unified framework that synthesis generation, exploration, and evaluation through step-level Monte Carlo Tree Search (MCTS) for efficient solution discovery. By integrating Alpha-UCT guided search into the interaction loop, Agent Alpha enables deliberate planning with early pruning of suboptimal branches and efficient prefix reuse. We further employ comparison-driven evaluation to mitigate absolute scoring biases and diversity-constrained expansion to maintain a compact, informative search space. We analyze the regret bound of Alpha-UCT and show it achieves tighter confidence bounds that accelerate identification of successful trajectories. On the OSWorld benchmark, Agent Alpha achieves a state-of-the-art success rate of ∼77%, significantly outperforming trajectory- level baselines. Code is available at: github.com/sizhe-tang/AgentAlpha.