作者位次 #3/5ACL 2026Poster· Natural Language Processing
H etero S pec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding
Siran Liu · Yang Ye · Qianchao Zhu · Zane Cao · Yongchao He
摘要
Autoregressive decoding inherently limits the inference throughput of Large Language Model (LLM) due to its sequential dependency. Speculative decoding mitigates this by verifying multiple predicted tokens in parallel, but its efficiency remains constrained by what we identify as verification het...