Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans
Suhong Moon · Minwoo Kang · Joseph Suh · Mustafa Safdari · John Canny
We present how base (pretrained) LMs conditioned via rich narrative backstories can faithfully simulate identity-driven human behavior and present analysis on the effects of contextual factors like question framing, time period, and participant pool.
摘要
Humans choose how to act via a nuanced process that depends both on rational deliberation and also identity and contextual factors. In this work, we study how large language models (LLMs) might simulate human action in the context of social dilemma games. While prior work has focused on ``steering'' (weak binding) of agent models to simulate personas, we analyze here how deep binding of base models with extended backstories leads to more faithful replication of identity-based behaviors. Our study has these findings: simulation fidelity vs human studies is improved by conditioning base LMs with a rich context of narrative identities and checking consistency using instruction-tuned models. We show that LLMs can also model contextual factors such as time (year that a study was performed) and Question Framing effects. LLMs, therefore allow us to explore the details that affect human studies but which are often omitted from experiment descriptions, and which prevent accurate replication.