Beyond Query Memorization: Large Language Model Routing with Query Decomposition and Historical Matching
Bo Lv · Jingbo Sun · Jianwei Lv · Chen Tang · Shaojie Zhang · Nayu Liu · Guoxin Yu · Zihao Li · Qichao Zhang · Dongbin Zhao · Ping Luo · Yue Yu
摘要
Optimizing the trade-off among predictive performance and computational cost is a central focus in the deployment of Large Language Models (LLMs). Current routing methods primarily rely on direct mapping from queries to models based on surface-level features, making them susceptible to the memori...