作者位次 #9/9ACL 2026Poster· Natural Language Processing
One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers
Diana Abagyan · Alejandro R. Salamanca · Andres Felipe Cruz-Salinas · Kris Cao · Hangyu Lin · Acyr Locatelli · Marzieh Fadaee · Ahmet Üstün · Sara Hooker
摘要
Pretraining massively multilingual Large Language Models (LLMs) for many languages at once is challenging due to limited model capacity, scarce high-quality data, and compute constraints. Moreover, the lack of language coverage in the tokenizer makes it harder to address the gap for new languages...