Acyr Locatelli

We introduce Tiny Aya, a family of open-weight multilingual language models that achieve strong, balanced performance across 70 languages with 3.35 billion parameters through deliberate data curation and training design. The release includes a pretrained base model, a globally balanced instruction-tuned variant, and three region-specialized models targeting Africa, South Asia, Europe, Asia-Pacific, and West Asia. We contribute: (1) a script-aware tokenizer weighting scheme that achieves competitive or superior compression across most writing systems, particularly for underrepresented scripts; (2) a posttraining pipeline combining FusioN, a multi-teacher synthetic data generation framework, with region-aware supervised finetuning and SimMerge, a predictive model merging method that preserves global consistency while retaining regional gains; (3) a comprehensive multilingual evaluation suite spanning translation, understanding, reasoning, generation, safety, and cultural awareness across up to 66 languages. Despite its compact size, Tiny Aya outperforms Gemma3-4B on translation in 46 of 55 languages, is competitive with or outperforms similar-sized models on generative benchmarks with substantially lower cross-language variance, and achieves the highest mean safe response rate (91.1%) on multilingual safety benchmarks. Quantized variants run on smartphones and quantization degradation attenuates for lower-resource languages . We release all models to support research into efficient, inclusive multilingual AI.

Pretraining massively multilingual Large Language Models (LLMs) for many languages at once is challenging due to limited model capacity, scarce high-quality data, and compute constraints. Moreover, the lack of language coverage in the tokenizer makes it harder to address the gap for new languages...