Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization
行走与言说:通过多模态代理策略优化弥合图像推理与行动之间的差距
机构 * National Key Laboratory for Novel Software Technology, Nanjing University, China(南京大学计算机软件新技术国家重点实验室) ; School of Artificial Intelligence, Nanjing University, China(南京大学人工智能学院) ; AI Business, Alibaba Group(阿里巴巴集团智能计算研究院) ; School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, China(上海交通大学自动化与智能感知学院) ; School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China(中山大学智能工程学院(深圳)) ; School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院) ; School of Software Technology, Zhejiang University, China(浙江大学软件学院) ; School of Computing, National University of Singapore, Singapore, Singapore(新加坡国立大学计算机学院)
AI总结 本文提出MAPO方法,通过强制生成视觉内容的显式文本描述,结合语义对齐与任务奖励,提升多模态推理能力,实验表明其在多个视觉推理基准上表现优异。