SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
Yuhao Shen · Junyi Shen · Quan Kong · Tianyu Liu · Yao Lu · Cong Wang
首次获取该候选人的完整档案及 StartLux 评估共 20 积分;每位用户单独解锁,之后查看免费。主动刷新 20 积分。
实时查找电子邮箱,费用不超过20积分;未找到邮箱不扣费。
Yuhao Shen · Junyi Shen · Quan Kong · Tianyu Liu · Yao Lu · Cong Wang
Yuhao Shen · Tianyu Liu · Junyi Shen · Jinyang Wu · Quan Kong · Huan Li · Cong Wang
Parallel Speculative Decoding (PSD) accelerates traditional Speculative Decoding (SD) by overlapping draft generation with verification. However, it remains hampered by two fundamental challenges: (1) a theoretical speedup ceiling dictated by the speed ratio between the draft and target models, a...
Tianyu Liu · Qitan Lv · Hao Li · Xing Gao · Xiao Sun · Xiaoyan Sun
Speculative decoding (SD), where a small draft model is employed to propose draft tokens in advance and then the target model validates them in parallel, has emerged as a promising technique for LLM inference acceleration. Many endeavors to improve SD are to eliminate the need for a draft model a...