SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
Keyang Xuan · Pengda Wang · Chongrui Ye · Haofei Yu · Tal August · Jiaxuan You
We propose a social learning environment that can probe agent social interaction performance under communication barrier scenarios
摘要
Large language models (LLMs) are increasingly evaluated in interactive environments to test their social intelligence. However, existing benchmarks often assume idealized communication, limiting our ability to diagnose whether LLMs can maintain and repair interactions under non-idealized communication conditions. To close this gap, we present a controlled interactive environment that can simulate social interaction under cognitive-difference-induced communication barriers. Grounded in a systematic literature review, \socialveil introduces three representative types of disruption, \emph{semantic vagueness}, \emph{sociocultural mismatch}, and \emph{emotional interference}. We also introduce two barrier-aware evaluation metrics, \emph{confusion resolution} and \emph{mutual understanding}, to evaluate interaction quality under impaired communication. Experiments across 720 scenarios and four frontier LLMs show that barriers consistently impair performance, with mutual understanding reduced by over 45\% on average, and confusion resolution reduced by nearly 50\%. Human evaluations validate the fidelity of these simulated barriers (ICC$\approx$0.78, Pearson r$\approx$0.80). We further demonstrate that adaptation strategies (Repair Instruction and Interactive learning) only have a modest effect far from barrier-free performance. This work takes a step toward bringing social interaction environments closer to real-world communication, opening opportunities for exploring the social intelligence of LLM agents.