一作ACL 2026Poster· Natural Language Processing
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
Jiandong Shao · Raphael Tang · Crystina Zhang · Karin Sevegnani · Pontus Stenetorp · Jianfei Yang · Yao Lu
摘要
Multilingual large language models achieve impressive cross-lingual performance despite largely monolingual pretraining. While bilingual data in pretraining corpora is widely believed to enable these abilities, details of its contributions remain unclear. We investigate this question by pretraini...