作者位次 #4/8ACL 2026Findings· Findings of ACL
What’s Missing in Screen-to-Action? Towards a UI -in-the-Loop Paradigm for Multimodal GUI Reasoning
Songze Li · Xiaoke Guo · Tianqi Liu · Biao Yi · Zhaoyan Gong · Zhiqiang Liu · Huajun Chen · Wen Zhang
摘要
Existing Graphical User Interface (GUI) reasoning tasks remain challenging, particularly in UI understanding. Current methods typically rely on direct screen-based decision-making, which lacks interpretability and overlooks a comprehensive understanding of UI elements, ultimately leading to task...