Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
Taiqiang Wu · Yuxin Cheng · Chenchen Ding · RUNMING YANG · Feng Xincheng · Wenyong Zhou · Zhengwu Liu · Ngai Wong
We first conduct a comprehensive investigation into the impact of noisy weights on LLM reasoning, and appraise three training-free strategies, including thinking mode, in-context learning, and module redundancy.
摘要
Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy efficiency and computational density. However, these architectures suffer from precision issues caused by intrinsic non-idealities of memristors. In this paper, we first conduct a comprehensive investigation into the impact of such typical non-idealities on LLM reasoning. Empirical results indicate that reasoning capability decreases significantly but varies for distinct benchmarks. Subsequently, we systematically appraise three training-free strategies, including thinking mode, in-context learning, and module redundancy. We thus summarize valuable guidelines, i.e., shallow layer redundancy is particularly effective for improving robustness, thinking mode performs better under low noise levels but degrades at higher noise, and in-context learning reduces output length with a slight performance trade-off. Our findings offer new insights into LLM reasoning under non-ideality and practical strategies to improve robustness.