Should We be Pedantic About Reasoning Errors in Machine Translation?
Calvin Bao · Marine Carpuat
easoning models make frequent errors when translating, but the errors are hard to detect reliably and fixing them rarely fixes the translation. We find that the trace isn't a faithful account of how the model translates.
摘要
Across seven language pairs (English $\rightarrow$ Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese), we show that large reasoning models (LRMs) exhibit reasoning errors during machine translation. We quantify these errors using an automated annotation protocol that labels reasoning steps as source-sentence misaligned, model-hypothesis misaligned, or reasoning-trace misaligned. We then intervene on candidate reasoning errors with weak-to-strong edits (hedging, removal, re-reasoning), and compare these interventions with new reasoning traces generated based on human translation references. Although reasoning errors can be detected with high precision in some language settings (e.g. Urdu), detection precision varies across languages and falls to 50\% in Spanish. Correcting error candidates in reasoning traces also fails to reliably improve translation quality, providing evidence that reasoning traces are not consistently faithful to the mechanisms underlying translation in LRMs.