VAPO : End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
Rui Hu · Delai Qiu · Yining Wang · Shengping Liu · Jitao Sang
摘要
Omni-modal large language models (OLLMs) offer a promising end-to-end solution for slide-enhanced speech recognition due to their inherent multimodal capabilities. However, we found a fundamental issue faced by OLLMs: Visual Interference , where models show a bias towards visible text over audito...