Researcher — paper.dinq.me

作者位次 #7/12ACL 2026Findings· Findings of ACL

Half- S : Halving the Scale for Near-Lossless 4-Bit LLM Training

Jinyang Du · Ruihao Gong · Linghan Ai · Zining Wang · Yunke Peng · Yao Wang · Lei YAN · wxuefei · Yaoyuan Wang · Jinyang Guo · Dahua Lin · Xianglong Liu

摘要

Training large language models (LLMs) at 4-bit precision offers substantial efficiency gains but remains challenging due to the limited dynamic range and coarse numerical resolution. Existing 4-bit training pipelines typically rely on max-scaling, which is ill-suited for heavy-tailed LLM tensor d...

效率与系统可解释性强化学习与后训练OpenReview 项目页

Lei YAN

获取完整研究档案

联系方式

2026 顶会论文

HiFloat4 Format for Language Model Pre-training on Ascend NPUs

Half- S : Halving the Scale for Near-Lossless 4-Bit LLM Training

常见合作者