Shujian Zhang

LLMs are deployed globally, yet produce responses biased towards cultures with abundant training data. Existing cultural localization approaches such as prompting or post-training alignment are black-box, hard to control, and do not reveal whether failures reflect missing knowledge or poor elicitation. In this paper, we address these gaps using mechanistic interpretability to uncover and manipulate cultural representations in LLMs. Leveraging sparse autoencoders (SAEs), we identify interpretable features that encode culturally salient information and aggregate them into $\textbf{Cu}$ltural $\textbf{E}$mbeddings ($\textit{CuE}$). We use $\textit{CuE}$ both to analyze implicit cultural biases given underspecified prompts, and to construct white-box interventions to steer responses towards a target culture. Using $\textit{CuE}$, we find that $\textbf{60\%}$ of the responses for $\textit{Gemma-2-9B}$ default to the United States (U.S.) and United Kingdom (U.K.). Across multiple models, we show that $\textit{CuE}$-based steering increases cultural faithfulness (win rates of $\textbf{48\%}$ vs. 24\% in pairwise comparisons) and elicits significantly rarer, long-tail cultural concepts than prompting alone (win rates of $\textbf{53\%}$ vs. 17\%). Notably, $\textit{CuE}$-based steering is complementary to black-box localization methods, offering gains when applied on top of prompt-augmented inputs. This also suggests that models can benefit by better elicitation strategies, and don't necessarily need to be fine-tuned with more data, though this varies across cultures.

To acquire instruction-following capabilities, large language models (LLMs) undergo instruction tuning, where they are trained on instruction-response pairs using next-token prediction (NTP). Efforts to improve instruction tuning often focus on higher-quality supervised fine-tuning (SFT) datasets...