Heng Ji

Recent advances in large language models (LLMs) have enabled strong performance in reasoning and environment-interaction, yet their ability for creative problem solving remains underexplored. One core form of such creativity is *tool use*: repurposing objects by reasoning about their affordances and attributes rather than relying on canonical usage. To study this capability, we introduce **CreativityBench**, a benchmark for affordance-based creative tool use. It is built on a large-scale affordance knowledge base containing 4K entities and 150K+ annotations linking objects, parts, attributes, and uses, from which we generate 14K grounded tasks requiring non-obvious but physically plausible solutions under constraints. Evaluation across 10 closed and open-source LLMs reveal a consistent gap: while models often select plausible objects, they fail to identify the correct parts, affordances, and underlying physical mechanism. Moreover, general reasoning ability does not reliably transfer to creative affordance discovery, gains from scaling quickly saturate, and Chain-of-Thought yields limited benefit. These findings highlight creative tool use as a missing dimension in current LLMs capabilities, and position CreativityBench as a systematic testbed for advancing creative intelligence in future LLMs and embodied agents.

Planning for real-world problems by language models often involves both world and user constraints, which may not be fully specified upfront and are progressively disclosed through interaction. However, existing benchmarks still underexplore adaptive planning under such progressively revealed dual constraints. To address this gap, we introduce AdaPlanBench, a dynamic interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed world and user constraints. AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints. At runtime, agents interact with the environment in a multi-turn protocol where hidden constraints are revealed only when the agent proposes a plan that violates them, requiring iterative plan revision under accumulating feedback. This makes planning challenging, as agents must infer and track constraints from feedback while re-planning effectively. Experiments on ten leading LLMs show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy. We further observe that performance degrades as more constraints accumulate, with user constraints posing a particularly large challenge and failures often stemming from weaker physical grounding and reduced effectiveness. These results establish AdaPlanBench as a testbed for dual-constrained interactive planning and highlight the challenge of reliable adaptation to dynamically revealed constraints in LLM agents.

Large Language Models (LLMs) exhibit strong generalist capabilities, yet task-specific behavior is often supported by diffuse and redundant structures. While pruning has been widely explored as a means to reduce inference costs, few existing methods translate theoretical sparsity into tangible speedups without sacrificing commensurate performance. We introduce **Sememe**, a task-aware depth-pruning method that frames block-importance estimation as causal inference, treating block removals as interventions in a Structural Causal Model (SCM) and estimating importance scores as path coefficients from per-example calibration losses, with a provable unbiasedness guarantee. Sememe identifies redundancy using a small calibration set and only the forward pass, avoiding the memory overhead of gradients or activations. Across Llama2, Llama3.1, Mistral, and Qwen3 models, Sememe with 32 calibration examples matches or exceeds the downstream retention that other training-free depth-pruning methods reach with 256-512 examples, while remaining stable where other methods degenerate. Sememe's interventional formulation naturally extends to joint multi-block removals and to task-aware calibration, which prunes toward a target skill from a small set of in-task examples. At larger scales, we find this depth pruning to be highly effective: Sememe retains 93.3\% of downstream accuracy when removing 20\% of Llama3.1-70B's layers, with speedups proportional to the fraction of removed layers on any hardware.

Academic benchmarks for coding agents tend to reward autonomous task completion, measured by verifiable rewards such as unit-test success. In contrast, real-world coding agents operate with humans in the loop, where success signals are typically noisy, delayed, and sparse. How can we bridge this gap? In this paper, we propose a process to learn a ``critic'' model from sparse and noisy interaction data, which can then be used for inference-time scaling and training-time data curation. Specifically, we introduce Critic Rubrics, a rubric-based supervision framework with 24 behavioral features that can be derived from human-agent interaction traces alone. Using a semi-supervised objective, we can then jointly predict these rubrics and sparse human feedback (when present). In experiments, we demonstrate that, despite being trained primarily from trace-observable rubrics and sparse real-world outcome proxies, these critics improve best-of-N reranking on SWE-bench (Best@8 +15.9 over Random@8 over the rerankable subset of trajectories), enable early stopping (+17.7 with 83\% fewer attempts), and support training-time data curation of trajectories.

Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current models struggle with accurate visual grounding and coherent reasoning chains. While extrinsic chain-of-thought prompting and visual cues significantly improve performance, current MLLMs lack intrinsic visual grounded reasoning capabilities, leading to inaccurate perception and reasoning disconnected from visual evidence. To address these limitations, we propose CURV, a curriculum learning framework that develops intrinsic visual reasoning capabilities by reformulating CQA as multi-step visual grounded reasoning, where each step coordinates logical reasoning with dynamic visual grounding through spatial attention concentration. To assist model learning, we further introduce CCQA, a three-level curriculum dataset with scalable synthetic generation across diverse chart types and reasoning patterns. Our curriculum systematically progresses from basic single-operation reasoning to complex multi-chart compositional tasks. Experiments demonstrate that CURV achieves up to $\uparrow20.50$% improvements over baselines and is generalizable to real-world benchmarks (up to $\uparrow12.30$%) and out-of-domain multimodal reasoning tasks (up to $\uparrow10.20$%), validating the effectiveness of internalizing visual reasoning with dynamic grounding for enhanced chart understanding capabilities.

Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains. We hypothesize that the success of such evolution frameworks hinges on meta-skills, such as self-reflection with environment feedback, that enable effective multi-round refinement, yet are largely neglected by traditional post-training. To bridge this gap, we present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning (RL), and inference-time evolutionary search. Concretely, we ground MetaEvolve in coding, where program execution provides natural, continuous reward signals beyond binary correctness. Building on these signals, we synthesize evolution trajectories as training data, each containing a current program, its fitness score (combining correctness and efficiency), and a history of prior attempts, and train the model via RL with verifiable rewards derived from test case execution. By training on large-scale code data, we aim to inspire generalizable domain-agnostic meta-skills that can transfer broadly to open-ended problems where such rich training signals are scarce. Across seven coding benchmarks, MetaEvolve outperforms the strongest baseline by 10.01\% absolute on in-distribution tasks and 24.12\% on out-of-distribution tasks. On open-ended algorithm optimization problems entirely outside the training domain, it further achieves a 46.9\% relative improvement. These results demonstrate that explicitly cultivating self-evolution meta-skills offers a principled path toward more capable and autonomously self-evolving AI.

We present Copyright Detective , the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyright infringement versus compliance as an evidence discovery process rather than a static classification task due to th...