一作ACL 2026Poster· Natural Language Processing
T i KM i X : Efficient Semi-Dynamic Data Mixture via Data Influence for LLM Pre-training
Yifan Wang · Binbinliu · Fengze Liu · Yuanfan Guo · Jiyao Deng · Xuecheng Wu · Weidong Zhou · Xiaohuan ZHOU · Taifeng Wang
摘要
The data mixture used in the pre-training of a language model is a cornerstone of its final performance. Static data mixing strategies in Large Language Model (LLM) pre-training are often suboptimal as they fail to adapt to the model’s evolving learning states. Conversely, fully online dynamic up...