一作ACL 2026Poster· Natural Language Processing
FLARE : Fine-Grained Length-Aware Routing for Resource-Efficient Heterogeneous LLM Serving
Yujia Fu · Heming Zhong · Dan Huang · Yutong Lu
摘要
With the rapid proliferation of large language models (LLMs), model pools have become increasingly heterogeneous in both capability and efficiency. Larger LLMs can improve quality but incur higher latency and cost, while smaller LLMs are the opposite, making per-query model selection crucial in p...