Anna Korhonen

Mechanistic interpretability has made it possible to localize circuits underlying specific behaviors in language models, but existing methods are expensive, model-specific, and difficult to scale to larger architectures. We introduce Differentiable Faithfulness Alignment (DFA), a framework for transferring circuit information from a smaller source model to a larger target model through a learned differentiable alignment. DFA maps source-model node importance scores into the target model and optimizes this mapping for causal faithfulness via a soft intervention objective, avoiding full circuit discovery on the target model. We evaluate DFA on Llama-3 and Qwen-2.5 across six tasks spanning factual retrieval, multiple-choice reasoning, and arithmetic. The strongest results occur on LLaMA-3-1B -> 3B, where aligned circuits are often competitive with direct target-model node attribution and zero-shot transfer remains effective. Recovery is weaker for larger source--target gaps and substantially worse on Qwen-2.5, suggesting that transfer becomes harder as architectural and scaling differences increase. Overall, DFA consistently improves over simple baselines and, in some settings, recovers target-model circuits with faithfulness comparable to direct attribution. These results suggest that smaller models can provide useful mechanistic priors for larger ones, while highlighting both the promise and the limits of node-level cross-model circuit alignment.

LLM evaluators (trained reward models and prompted LLM-as-a-Judge) are routinely validated via pairwise accuracy. In a multilingual setting, this operates under the premise that high pairwise accuracy implies reliable, language-neutral scoring. We show that this assumption does not hold across languages. We conduct experiments with semantically identical instruction--response pairs across 23 languages, and find that multilingual evaluators assign significantly different scores to different evaluation languages. We show that this bias is statistically significant and consistent across eight evaluators of different architectures and training paradigms, and is strongly correlated with language resource level: lower-resource languages are scored more generously. Meanwhile, these biases are invisible to pairwise accuracy: evaluators would achieve above 90\% pairwise accuracy, yet have 43% difference in acceptance-rate across languages under a global decision threshold, meaning, for instance, that harmful content in lower-resource languages is more likely to pass safety filters. We then investigate why lower-resource languages receive higher rather than lower scores, and we find that model uncertainty (measured by negative log-likelihood) is linked with the effect: models tend to give higher scores when less confident; however, language identity remains a significant predictor after controlling for uncertainty, and the bias cannot be explained away by content difficulty alone, but is a structural, language-level misalignment.