一作ACL 2026Findings· Findings of ACL
Half- S : Halving the Scale for Near-Lossless 4-Bit LLM Training
Jinyang Du · Ruihao Gong · Linghan Ai · Zining Wang · Yunke Peng · Yao Wang · Lei YAN · wxuefei · Yaoyuan Wang · Jinyang Guo · Dahua Lin · Xianglong Liu
摘要
Training large language models (LLMs) at 4-bit precision offers substantial efficiency gains but remains challenging due to the limited dynamic range and coarse numerical resolution. Existing 4-bit training pipelines typically rely on max-scaling, which is ill-suited for heavy-tailed LLM tensor d...