PDT rim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
Hao Zhang · Lyu Mengsi · Zhuo Chen · Yulong Ao · Yonghua Lin
摘要
Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning provides an effective means to alleviate these demands. However, existing methods often ignore the characteristics of p...