I nference D ynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling
Haochen Shi · Tianshi Zheng · Weiqi Wang · Baixuan Xu · Chunyang Li · Chunkit Chan · Tao Fan · Yangqiu Song
摘要
Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs, enabling the selection of the best-performing LLMs for specific user queries while balancing performance and cost. However, current routing approaches often face limitations in scalability when d...