A rc L ight: A Lightweight LLM Inference Architecture for Many-Core CPU s
Yuzhuang Xu · Xu Han · Yuxuan Li · Wanxiang Che
摘要
Although existing frameworks for large language model (LLM) inference on CPUs are mature, they fail to fully exploit the computational potential of many-core CPU platforms. Many-core CPUs are widely deployed in web servers and high-end networking devices, and are typically organized into multiple...