Marine Carpuat

Across seven language pairs (English $\rightarrow$ Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese), we show that large reasoning models (LRMs) exhibit reasoning errors during machine translation. We quantify these errors using an automated annotation protocol that labels reasoning steps as source-sentence misaligned, model-hypothesis misaligned, or reasoning-trace misaligned. We then intervene on candidate reasoning errors with weak-to-strong edits (hedging, removal, re-reasoning), and compare these interventions with new reasoning traces generated based on human translation references. Although reasoning errors can be detected with high precision in some language settings (e.g. Urdu), detection precision varies across languages and falls to 50\% in Spanish. Correcting error candidates in reasoning traces also fails to reliably improve translation quality, providing evidence that reasoning traces are not consistently faithful to the mechanisms underlying translation in LRMs.

Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited by the lack of reliable evaluation. We investigate whether existing automatic evaluation metrics developed for modern languages are reliable in this setting, using translation from Classical Chinese to English as a test case. We introduce a diagnostic framework based on minimal pairs capturing error types salient in scholarly use, probing both reference-based and reference-free metrics for error sensitivity and tolerance to valid variation. We find that all metrics exhibit blind spots, however MetricX24 performs best overall. Our findings highlight the need for more robust and interpretable metrics for historically and culturally distinct translation settings.

Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps can be closed simply by making reasoning in every language resemble English reasoning. This work challenges this assumption by asking instead: what actually characterizes successful reasoning traces in multilingual settings, and to what extent do English‑derived reasoning features genuinely help in other languages? We first define a suite of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces, and use logistic regression to quantify how each feature associates with final answer accuracy. We further train sparse autoencoders over multilingual traces to automatically discover latent reasoning concepts that instantiate or extend these features. Finally, we use the features to re-rank traces and measure their impact on accuracy at test time. Across two mathematical reasoning benchmarks, four LRMs, and ten languages, we find that most features are positively associated with accuracy, but the strength of association varies considerably across languages and can even reverse in some. Our findings challenge English-centric reward designs and point toward adaptive objectives that accommodate language-specific reasoning patterns, with concrete implications for multilingual benchmark and reward design.