视觉大模型 / VLM

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

Xun Zhu, Ying Hu, Fanbin Mo, Miao Li, Ji Wu

专题命中 视觉问答 ：visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale

Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu, Xiang Wan, Benyou Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Explore until Confident: Efficient Exploration for Embodied Question Answering

Allen Z. Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, Dorsa Sadigh

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Robotics: Science and Systems (RSS) 2024

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, Zhuowen Tu

专题命中 视觉问答 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at AAAI Conference on Artificial Intelligence (AAAI-24)

RoboVQA: Multimodal Long-Horizon Reasoning for Robotics

Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, Pete Florence, Wei Han, Robert Baruch, Yao Lu, Suvir Mirchandani, Peng Xu, Pannag Sanketi, Karol Hausman, Izhak Shafran, Brian Ichter, Yuan Cao

专题命中 视觉问答 ：VLM(abstract)；visual language model(abstract)；visual question answering(abstract)

Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Zhu Wang, Sourav Medya, Sathya N. Ravi

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning

Qian Jiang, Changyou Chen, Han Zhao, Liqun Chen, Qing Ping, Son Dinh Tran, Yi Xu, Belinda Zeng, Trishul Chilimbi

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 14 pages, 8 figure, CVPR 2023 accepted

Visually Grounded VQA by Lattice-based Retrieval

Daniel Reich, Felix Putze, Tanja Schultz

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives

Zhuofan Ying, Peter Hase, Mohit Bansal

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2022 (first two authors contributed equally)

SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency

Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Parikh, Ramprasaath R. Selvaraju

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to the NeurIPS 2020 workshop on Interpretable Inductive Biases and Physically Structured Learning

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2020

Neural Reasoning, Fast and Slow, for Video Question Answering

Thao Minh Le, Vuong Le, Svetha Venkatesh, Truyen Tran

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref International Joint Conference on Neural Networks (IJCNN) 2020

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images

从像素到感受：将MLLMs对齐于人类对图像的认知感知

Yiming Chen, Junlin Han, Tianyi Bai, Shengbang Tong, Filippos Kokkinos, Philip Torr

机构 * Oxford University（牛津大学） ； HKUST（香港科技大学） ； University College London（伦敦大学学院） ； New York University（纽约大学） 纠错

专题命中 视觉问答 ：MLLM(abstract,comments)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

AI总结 本文提出CogIP-Bench基准，通过后训练提升MLLMs对图像认知属性的对齐能力，并展示其在图像生成中的应用。

Comments Project page with codes/datasets/models: https://follen-cry.github.io/MLLM-Cognition-project-page/

Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA

Itbaan Safwan, Muhammad Annas Shaikh, Muhammad Haaris, Ramail Khan, Muhammad Atif Tahir

机构 * School of Mathematics and Computer Science, Institute of Business Administration (IBA), Karachi, Pakistan（数学与计算机科学学院，商学院（IBA），巴基斯坦卡里奇） 纠错

专题命中 视觉问答 ：visual question answering(abstract,comments)；grounding(abstract)；分类 cs.CV、cs.LG

Comments This is a working paper submitted for Medico 2025: Visual Question Answering (with multimodal explanations) for Gastrointestinal Imaging at MediaEval 2025. 5 pages, 3 figures and 1 table

CrossView: Can Vision-Language Models Reason Across Cameras?

CrossView：视觉-语言模型能否跨相机进行推理？

Sahil Shah, S P Sharan, Harsh Goel, Manvik Pasula, Adithya Hebbalae, Minkyu Choi, Sandeep P. Chinchali

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

AI总结 本文提出CrossView多相机视频问答基准，评估发现GPT-5.2等模型跨相机推理准确率低，开源模型表现更差，该基准可用于测试模型联合处理多视角的能力。

Comments ECCV 2026

Binary Tracking for Spatial QA and Navigation with Open Vision-Language Models

基于开放视觉语言模型的空间问答与导航的二值追踪

Dongbin Na, Chanwoo Kim, Soonbin Rho, Giyun Choi, Gangbok Lee, Dooyoung Hong

机构 * RGA Inc.（RGA公司） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.AI

AI总结 提出BinTrack，一种全开源的空间定位代理，通过二值搜索轨迹段，在SpaceLocQA基准上准确率提升22.8%，推理速度提升1.5倍，并发布多行程室外数据集GangnamLoop。

Comments 21 pages, 4 figures, 15 tables. Project page: https://ndb796.github.io/BinaryTracking ; Code and dataset: https://github.com/ndb796/BinaryTracking

A Comprehensive Survey of Knowledge-Based Vision Question Answering Systems: The Lifecycle of Knowledge in Visual Reasoning Task

基于知识的视觉问答系统综述：视觉推理任务中的知识生命周期

Jiaqi Deng, Zonghan Wu, Huan Huo, Guandong Xu

机构 * University Technology of Sydney（悉尼大学技术学院） ； East China Normal University（华东师范大学） ； Education University of Hong Kong（香港教育大学） 纠错

专题命中 视觉问答 ：visual reasoning(title)；分类 cs.CV

AI总结 综述基于知识的视觉问答（KB-VQA）方法，将其分为知识表示、检索和推理三个阶段，并探讨大语言模型带来的变革，指出未来研究方向。

Comments Accepted at TKDE, 20 pages, 5 figures, 4 tables

Journal ref IEEE Transactions on Knowledge and Data Engineering, 2026

Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusion

Mind-Omni：通过离散扩散实现脑-视觉-语言建模的统一多任务框架

Yizhuo Lu, Changde Du, Qingyu Shi, Hang Chen, Jie Peng, Liuyun Jiang, Shuangchen Zhao, Huiguang He

机构 * NeuBCI Lab, State Key Laboratory of Brain Cognition ； Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China ； School of Future Technology, University of Chinese Academy of Sciences, Beijing, China ； School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China ； Zhongguancun Academy, Beijing, China ； Peking University, Beijing, China 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.AI

AI总结 提出Mind-Omni框架，利用离散扩散范式统一七种编码与解码任务，通过脑分词器将连续脑信号转化为离散令牌，实现多模态交互，并构建脑问答指令调优数据集，在多项任务上达到或超越专用模型性能。

Can Vision-Language Models Answer Face to Face Questions in the Real-World?

视觉-语言模型能否在现实世界中回答面对面问题？

Reza Pourreza, Rishit Dagli, Apratim Bhattacharyya, Sunny Panchal, Guillaume Berger, Roland Memisevic

机构 * Qualcomm AI Research（高通人工智能研究） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

AI总结 本文提出了一种新的数据集IVD，用于评估视觉-语言模型在现实世界中回答面对面问题的能力，并通过微调减少与人类表现的差距。

Comments ICLR 2026 paper

IRIS: Intent Resolution via Inference-time Saccades for Open-Ended VQA in Large Vision-Language Models

IRIS：通过推理时的注视来解决大型视觉-语言模型中开放式视觉问答的意图

Parsa Madinei, Srijita Karmakar, Russell Cohen Hoffing, Felix Gervitz, Miguel P. Eckstein

机构 * Department of Computer Science, UC Santa Barbara（计算机科学系，加州大学圣芭芭拉分校） ； Department of Psychological & Brain Sciences, UC Santa Barbara（心理学与脑科学系，加州大学圣芭芭拉分校） ； DEVCOM Army Research Laboratory（国防部陆军研究实验室） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

AI总结 IRIS通过实时眼动数据提升大型视觉-语言模型在开放式视觉问答中的意图解析准确率

Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs

从医学文献合成高质量的视觉问答系统：基于生成-验证框架的大型多模态模型

Xiaoke Huang, Ningsen Wang, Hui Liu, Xianfeng Tang, Yuyin Zhou

机构 * UC Santa Cruz（加州大学圣克ruz分校） ； Fudan University（复旦大学） ； Amazon Research（亚马逊研究） 纠错

专题命中 视觉问答 ：visual question answering(title)；分类 cs.LG

AI总结 MedVLSynther通过生成-验证框架从开放文献合成高质量医学VQA数据，提升六个基准测试的准确率，达到77.57的VQA-RAD表现。

Comments Project page, code, data, and models: https://ucsc-vlaa.github.io/MedVLSynther/ ; Accepted by ICLR'26

Towards Understanding Best Practices for Quantization of Vision-Language Models

朝着理解视觉-语言模型量化最佳实践

Gautom Das, Vincent La, Ethan Lau, Abhinav Shrivastava, Matthew Gwilliam

机构 * University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

AI总结 本研究探讨了视觉-语言模型量化方法对多模态流水线性能的影响，发现LLM低位量化在减少bpw时仍保持高精度，为高效部署MLLMs提供实践指导。

Comments 15 pages, 12 figures, 1 table

MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering

MAVIS：多模态来源归因的长形式视觉问答基准

Seokwon Song, Minsu Park, Gunhee Kim

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV

AI总结 MAVIS基准旨在评估多模态来源归因系统，通过多模态文档检索和长形式答案生成，揭示多模态设置下信息量、 groundedness 和流畅性之间的权衡与改进方向。

Comments AAAI 2026; code is available at https://github.com/seokwon99/MAVIS

Mined Prompting and Metadata-Guided Generation for Wound Care Visual Question Answering

Bavana Durgapraveen, Sornaraj Sivasankaran, Abhinand Balachandran, Sriram Rajkumar

机构 * EXL Health AI Lab at MEDIQA-WV 2025（EXL健康AI实验室） 纠错

专题命中 视觉问答 ：visual question answering(title)；分类 cs.AI

Comments 2 figures, 11 pages

Lost in Embeddings: Information Loss in Vision-Language Models

Wenyan Li, Raphael Tang, Chengzu Li, Caiqi Zhang, Ivan Vulić, Anders Søgaard

机构 * University of Copenhagen（哥本哈根大学） ； Microsoft（微软） ； University of Cambridge（剑桥大学） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

ConSens: Assessing context grounding in open-book question answering

Ivan Vankov, Matyo Ivanov, Adriana Correia, Victor Botev

机构 * Iris.ai BG 纠错

专题命中 视觉问答 ：grounding(title)；分类 cs.LG

Comments 9 pages, 3 figures, 3 tables

Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform

Cheonsu Jeong

专题命中 视觉问答 ：multimodal large language model(title)；分类 cs.AI

Comments 22 pages, 27 figures

Journal ref 2025 Journal of Intelligence and Information Systems

Patch-level Sounding Object Tracking for Audio-Visual Question Answering

Zhangbin Li, Jinxing Zhou, Jing Zhang, Shengeng Tang, Kun Li, Dan Guo

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV

Comments Accepted by AAAI 2025

ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

Nghia Hieu Nguyen, Tho Thanh Quan, Ngan Luu-Thuy Nguyen

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV

Comments PACLIC 2024

StackOverflowVQA: Stack Overflow Visual Question Answering Dataset

Motahhare Mirzaei, Mohammad Javad Pirhadi, Sauleh Eetemadi

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV