一作ACL 2026Findings· Findings of ACL
RACER : Retrieval-Augmented Contextual Rapid Speculative Decoding
Zihong Zhang · Zuchao Li · Lefei Zhang · Ping Wang · Hai Zhao
摘要
Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigates this through a guess-and-verify strategy, but existing training-free variants face trade-offs: retrieval-based drafts break when no exact match...