作者位次 #3/20OSDI 2026OralBest Paper· Operating Systems & Distributed Systems
Teaching the Old Dog New Tricks: Building Efficient Data Pipelines for Large-Scale LLM Pre-Training (Operational Systems)
Luofan Chen · Chenhan Wang · Weidong Zhang · Jinxin Chi · Hequan Zhang · Zanbo Wang · Chenyuan Wang · Lishu Luo · Sijin Wu · Junqi Hu · Jun Wang · Cheng Chen · Lixin Huang · Liyang Zhao · Yong Tian · Jun Guo · Youhui Bai · Wencong Xiao · Kang Chen · Cheng Li
摘要
Data pipelines play a critical role in the performance of large-scale pre-training jobs running on thousands of GPUs. In this work, we present a comprehensive quantitative analysis of data access patterns from production workloads and reveal three previously underreported bottlenecks. First, cros...