Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement
Qimin Zhong · Hao Liao · Haiming Qin · Mingyang Zhou · Rui Mao · Wei Chen · Naipeng Chao
摘要
Whether Large Language Models (LLMs) develop coherent internal world models remains a core debate. While conventional Next-Token Prediction (NTP) focuses on one-step-ahead supervision, Multi-Token Prediction (MTP) has shown promise in learning more structured representations. In this work, we pro...