作者位次 #5/9OSDI 2026Oral· Operating Systems & Distributed Systems
Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
Haoyang Li · Fangcheng Fu · Hao Ge · Sheng Lin · Xuanyu Wang · Jiawen Niu · Yuming Zhou · Xupeng Miao · Bin CUI
摘要
The Single-Program Multiple-Data (SPMD) paradigm provides a unified abstraction to annotate various parallel dimensions in distributed deep learning (DL) training. With SPMD, users can write training programs from the viewpoint of a single device, and the system will automatically deduce the tens...