Christof Monz

Reinforcement learning for large language model reasoning often suffers from signal collapse: under uniform small-group sampling, difficult prompts frequently produce uninformative all-fail or all-success groups, causing useful gradients to disappear. We show that this failure is largely a statistical artifact of undersampling rather than an inherent limitation of the model. This suggests that sampling budget should be allocated adaptively across prompts according to their pass rates. We formalize this idea through a family of non-linear RL objectives, which induce difficulty-aware weighting and recover prior log-objective analyses as a special case. Based on this view, we propose Reinforce-Ada, with an estimation-based variant and a model-free sequential variant that adaptively allocate rollouts to recover informative signals. Across multiple models and math reasoning benchmarks, Reinforce-Ada consistently improves over standard GRPO baselines, recovering much of the benefit of large-group training at substantially lower cost.

We investigate the separation of literal interpretation from contextual inference in a collaborative block-building tasks, where an agent must resolve underspecified instructions using context. We adapt an existing two-speaker psycholinguistic paradigm into an interactive benchmark called Build What I Mean (BWIM). This setup contrasts a pragmatically cooperative speaker with one who is only literally reliable. In BWIM, models face underspecified instructions and must choose between making a contextual inference or requesting clarification at a small communication cost. Evaluating several state-of-the-art LLMs, we find a clear dissociation between judgment and action. Although models successfully detect speaker unreliability in explicit confidence ratings, they fail to leverage this awareness when taking action. Instead of deploying efficient clarification strategies, models default to suboptimal behaviors. These include partner-blind over-clarification and question-averse guessing under uncertainty. BWIM provides a controlled environment to evaluate online partner adaptation and contextual reasoning in interactive settings.

Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining. Similarly, Chain-of-Thought (CoT) prompting and its extension, Long CoT, improve accuracy by generating rich intermediate reasoning trajectories, but these approaches incur substantial token costs that impede their deployment in latency-sensitive settings. In this work, we first show that truncated CoT, which stops reasoning before completion and directly generates the final answer, often matches full CoT sampling while using dramatically fewer tokens. Building on this insight, we introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling along three orthogonal axes: (1) the number of reasoning trajectories, (2) the number of final solutions per trajectory, and (3) the depth at which reasoning traces are truncated. Through extensive experiments on five diverse reasoning benchmarks and several model scales, we demonstrate that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget. Our analysis reveals how to allocate computation across these dimensions to maximize performance, paving the way for more efficient and scalable LLM reasoning.