作者位次 #2/4ACL 2026Findings· Findings of ACL
Weight Tying Biases Token Embeddings Towards the Output Space
Antonio Lopardo · Avyukth Harish · Catherine Arnett · Akshat Gupta
摘要
Weight tying, i.e. sharing parameters between input and output embedding matrices, is common practice in language model design, yet its impact on the learned embedding space remains poorly understood. In this paper, we show that tied embedding matrices align more closely with output (unembedding)...