一作ACL 2026Findings· Findings of ACL
Discovering the Gems in Early Layers: Accelerating Long-Context LLM s with 1000x Input Token Reduction
Zhenmei Shi · Yifei Ming · Xuan-Phi Nguyen · Yingyu Liang · Shafiq Joty
摘要
Large Language Models (LLMs) have demonstrated remarkable capabilities in handling long context inputs, but this comes at the cost of increased computational resources and latency. Our research introduces a novel approach for the long context bottleneck to accelerate LLM inference and reduce GPU...