Activation Steering via Generative Causal Mediation
Aruna Sankaranarayanan · Amir Zur · Atticus Geiger · Dylan Hadfield-Menell
We localize concepts from long-form response using causal mediation
摘要
A central goal of interpretability research is to isolate model components that are responsible for specific behaviors. In practice, localization methods rely on the likelihood of a single token or the likelihood differences between matched tokens in otherwise identical responses. This limits the range of behaviors these methods can study. To overcome this limitation, we introduce Generative Causal Mediation (GCM), which extends localization methods to free-form responses. GCM uses a dataset of contrastive prompt pairs and the associated unconstrained responses to identify model components whose activations best explain the difference in the responses. We apply GCM to localize refusal, sycophancy, and style transfer in OLMo, Qwen, and SOLAR LMs. Compared to linear probe and random baselines, GCM-ranked attention heads reach a given steering success rate while intervening on fewer heads, and generalize better to unseen prompts. A linear approximation of GCM based on attribution patching performs comparably at a fraction of the compute. Our results extend the range of behaviors that can be studied with causal mediation.