“I Didn’t Make the Micro Decisions”: Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration
Eunsu Kim · Jessica R. Mindel · Kyungjin Kim · Tongshuang Wu
We introduce a goal-level attribution framework that shifts the focus from goal execution to goal shaping.
摘要
As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human–AI collaboration becomes critical—both for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, COTRACE, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying COTRACE to 638 real-world collaboration logs, we find that while models account for only ~10% of goal-shaping at the macro level, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior, with downstream effects on output diversity. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.