Joyce Chai

Large language model (LLM) agents are increasingly used for goal-directed tasks, yet collaboration under asymmetric information remains underexplored. We adapt Einstein Puzzles into a tabletop game in which two agents hold complementary constraints and control different parts of the workspace. Solving each game requires agents to reason, communicate, and manipulate objects toward a shared goal. We fine-tune agents under four communicative action spaces and evaluate an environment-based reasoning verifier that filters candidate actions using physical affordances, communication history, and inferred constraints. In the strongest Llama 3.1 configuration, agents that can both provide and seek information achieve the highest success rate, while mismatched communication capabilities reduce performance. Verifier-assisted inference improves success rates across all evaluated action spaces. Some variants also achieve high success without communication, but their trajectories contain more rule-inconsistent actions and rejected placement attempts. In a human study with 30 participants, non-communicative agents require a similar number of steps as fully communicative agents but are rated as more confusing. These findings show that task success alone does not capture the quality of collaborative behavior and highlight the value of bidirectional communication and environment-derived verification.