一作OSDI 2026Oral· Operating Systems & Distributed Systems
ADAngel: Accelerating Arbitrary-Precision Quantized LLMs with Adaptive Computing Mapping
Yao Liu · Wenjie Wang · Yifei Feng · Bo Peng · Jianguo Yao · Haibing Guan
摘要
Arbitrary-Precision Quantization (APQ), which uses asymmetric bit-widths for weights and activations (e.g., W4A8), is a prevalent technique for LLM inference because of its excellent accuracy-performance balance. APQ transforms the general matrix multiplications (GEMM), the core of LLM computatio...