视觉大模型 / VLM - arXivDaily 专题

2308.09936 2023-12-19 cs.CV cs.AI cs.CL cs.LG 75%

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, Zhuowen Tu

专题命中视觉问答：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at AAAI Conference on Artificial Intelligence (AAAI-24)

2311.00899 2023-11-03 cs.RO 75%

RoboVQA: Multimodal Long-Horizon Reasoning for Robotics

Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, Pete Florence, Wei Han, Robert Baruch, Yao Lu, Suvir Mirchandani, Peng Xu, Pannag Sanketi, Karol Hausman, Izhak Shafran, Brian Ichter, Yuan Cao

专题命中视觉问答：VLM(abstract)；visual language model(abstract)；visual question answering(abstract)

2302.05608 2023-10-09 cs.CV cs.AI cs.CL cs.LG 75%

Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Zhu Wang, Sourav Medya, Sathya N. Ravi

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

2303.05952 2023-03-13 cs.LG cs.AI cs.CV 75%

Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning

Qian Jiang, Changyou Chen, Han Zhao, Liqun Chen, Qing Ping, Son Dinh Tran, Yi Xu, Belinda Zeng, Trishul Chilimbi

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 14 pages, 8 figure, CVPR 2023 accepted

2211.08086 2022-11-16 cs.CV cs.AI cs.CL cs.HC cs.LG 75%

Visually Grounded VQA by Lattice-based Retrieval

Daniel Reich, Felix Putze, Tanja Schultz

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

2206.11212 2022-10-27 cs.CV cs.AI cs.CL cs.LG 75%

VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives

Zhuofan Ying, Peter Hase, Mohit Bansal

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2022 (first two authors contributed equally)

2010.10038 2020-12-02 cs.CV cs.AI cs.CL cs.LG 75%

SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency

Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Parikh, Ramprasaath R. Selvaraju

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to the NeurIPS 2020 workshop on Interpretable Inductive Biases and Physically Structured Learning

2009.11278 2020-09-24 cs.CV cs.AI cs.CL cs.LG 75%

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2020

1907.04553 2020-04-14 cs.CV cs.AI cs.LG 75%

Neural Reasoning, Fast and Slow, for Video Question Answering

Thao Minh Le, Vuong Le, Svetha Venkatesh, Truyen Tran

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref International Joint Conference on Neural Networks (IJCNN) 2020

2511.22805 2025-12-01 cs.CV cs.LG cs.MM 74%

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images

从像素到感受：将MLLMs对齐于人类对图像的认知感知

Yiming Chen, Junlin Han, Tianyi Bai, Shengbang Tong, Filippos Kokkinos, Philip Torr

机构 * Oxford University（牛津大学）； HKUST（香港科技大学）； University College London（伦敦大学学院）； New York University（纽约大学）

专题命中视觉问答：MLLM(abstract,comments)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

AI总结本文提出CogIP-Bench基准，通过后训练提升MLLMs对图像认知属性的对齐能力，并展示其在图像生成中的应用。

Comments Project page with codes/datasets/models: https://follen-cry.github.io/MLLM-Cognition-project-page/

2511.04384 2025-11-07 cs.CV cs.LG 74%

Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA

Itbaan Safwan, Muhammad Annas Shaikh, Muhammad Haaris, Ramail Khan, Muhammad Atif Tahir

机构 * School of Mathematics and Computer Science, Institute of Business Administration (IBA), Karachi, Pakistan（数学与计算机科学学院，商学院（IBA），巴基斯坦卡里奇）

专题命中视觉问答：visual question answering(abstract,comments)；grounding(abstract)；分类 cs.CV、cs.LG

Comments This is a working paper submitted for Medico 2025: Visual Question Answering (with multimodal explanations) for Gastrointestinal Imaging at MediaEval 2025. 5 pages, 3 figures and 1 table

2608.15539 2026-08-18 cs.CV 新提交 74%

CrossView: Can Vision-Language Models Reason Across Cameras?

CrossView：视觉-语言模型能否跨相机进行推理？

Sahil Shah, S P Sharan, Harsh Goel, Manvik Pasula, Adithya Hebbalae, Minkyu Choi, Sandeep P. Chinchali

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校）

专题命中视觉问答：vision-language model(title)；分类 cs.CV

AI总结本文提出CrossView多相机视频问答基准，评估发现GPT-5.2等模型跨相机推理准确率低，开源模型表现更差，该基准可用于测试模型联合处理多视角的能力。

Comments ECCV 2026

2606.16902 2026-06-16 cs.RO cs.AI 新提交 74%

Binary Tracking for Spatial QA and Navigation with Open Vision-Language Models

基于开放视觉语言模型的空间问答与导航的二值追踪

Dongbin Na, Chanwoo Kim, Soonbin Rho, Giyun Choi, Gangbok Lee, Dooyoung Hong

机构 * RGA Inc.（RGA公司）

专题命中视觉问答：vision-language model(title)；分类 cs.AI

AI总结提出BinTrack，一种全开源的空间定位代理，通过二值搜索轨迹段，在SpaceLocQA基准上准确率提升22.8%，推理速度提升1.5倍，并发布多行程室外数据集GangnamLoop。

Comments 21 pages, 4 figures, 15 tables. Project page: https://ndb796.github.io/BinaryTracking ; Code and dataset: https://github.com/ndb796/BinaryTracking

2504.17547 2026-06-16 cs.CV cs.IR cs.MM 版本更新 74%

A Comprehensive Survey of Knowledge-Based Vision Question Answering Systems: The Lifecycle of Knowledge in Visual Reasoning Task

基于知识的视觉问答系统综述：视觉推理任务中的知识生命周期

Jiaqi Deng, Zonghan Wu, Huan Huo, Guandong Xu

机构 * University Technology of Sydney（悉尼大学技术学院）； East China Normal University（华东师范大学）； Education University of Hong Kong（香港教育大学）

专题命中视觉问答：visual reasoning(title)；分类 cs.CV

AI总结综述基于知识的视觉问答（KB-VQA）方法，将其分为知识表示、检索和推理三个阶段，并探讨大语言模型带来的变革，指出未来研究方向。

Comments Accepted at TKDE, 20 pages, 5 figures, 4 tables

Journal ref IEEE Transactions on Knowledge and Data Engineering, 2026

2605.29591 2026-05-29 cs.AI 74%

Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusion

Mind-Omni：通过离散扩散实现脑-视觉-语言建模的统一多任务框架

Yizhuo Lu, Changde Du, Qingyu Shi, Hang Chen, Jie Peng, Liuyun Jiang, Shuangchen Zhao, Huiguang He

机构 * NeuBCI Lab, State Key Laboratory of Brain Cognition ； Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China ； School of Future Technology, University of Chinese Academy of Sciences, Beijing, China ； School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China ； Zhongguancun Academy, Beijing, China ； Peking University, Beijing, China

专题命中视觉问答：vision-language model(title)；分类 cs.AI

AI总结提出Mind-Omni框架，利用离散扩散范式统一七种编码与解码任务，通过脑分词器将连续脑信号转化为离散令牌，实现多模态交互，并构建脑问答指令调优数据集，在多项任务上达到或超越专用模型性能。

2503.19356 2026-02-24 cs.CV 74%

Can Vision-Language Models Answer Face to Face Questions in the Real-World?

视觉-语言模型能否在现实世界中回答面对面问题？

Reza Pourreza, Rishit Dagli, Apratim Bhattacharyya, Sunny Panchal, Guillaume Berger, Roland Memisevic

机构 * Qualcomm AI Research（高通人工智能研究）

专题命中视觉问答：vision-language model(title)；分类 cs.CV

AI总结本文提出了一种新的数据集IVD，用于评估视觉-语言模型在现实世界中回答面对面问题的能力，并通过微调减少与人类表现的差距。

Comments ICLR 2026 paper

2602.16138 2026-02-19 cs.CV 74%

IRIS: Intent Resolution via Inference-time Saccades for Open-Ended VQA in Large Vision-Language Models

IRIS：通过推理时的注视来解决大型视觉-语言模型中开放式视觉问答的意图

Parsa Madinei, Srijita Karmakar, Russell Cohen Hoffing, Felix Gervitz, Miguel P. Eckstein

机构 * Department of Computer Science, UC Santa Barbara（计算机科学系，加州大学圣芭芭拉分校）； Department of Psychological & Brain Sciences, UC Santa Barbara（心理学与脑科学系，加州大学圣芭芭拉分校）； DEVCOM Army Research Laboratory（国防部陆军研究实验室）

专题命中视觉问答：vision-language model(title)；分类 cs.CV

AI总结 IRIS通过实时眼动数据提升大型视觉-语言模型在开放式视觉问答中的意图解析准确率

2510.25867 2026-02-19 cs.LG 74%

Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs

从医学文献合成高质量的视觉问答系统：基于生成-验证框架的大型多模态模型

Xiaoke Huang, Ningsen Wang, Hui Liu, Xianfeng Tang, Yuyin Zhou

机构 * UC Santa Cruz（加州大学圣克ruz分校）； Fudan University（复旦大学）； Amazon Research（亚马逊研究）

专题命中视觉问答：visual question answering(title)；分类 cs.LG

AI总结 MedVLSynther通过生成-验证框架从开放文献合成高质量医学VQA数据，提升六个基准测试的准确率，达到77.57的VQA-RAD表现。

Comments Project page, code, data, and models: https://ucsc-vlaa.github.io/MedVLSynther/ ; Accepted by ICLR'26

2601.15287 2026-01-22 cs.CV 74%

Towards Understanding Best Practices for Quantization of Vision-Language Models

朝着理解视觉-语言模型量化最佳实践

Gautom Das, Vincent La, Ethan Lau, Abhinav Shrivastava, Matthew Gwilliam

机构 * University of Maryland, College Park（马里兰大学学院公园分校）

专题命中视觉问答：vision-language model(title)；分类 cs.CV

AI总结本研究探讨了视觉-语言模型量化方法对多模态流水线性能的影响，发现LLM低位量化在减少bpw时仍保持高精度，为高效部署MLLMs提供实践指导。

Comments 15 pages, 12 figures, 1 table

2511.12142 2025-12-19 cs.CV 74%

MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering

MAVIS：多模态来源归因的长形式视觉问答基准

Seokwon Song, Minsu Park, Gunhee Kim

专题命中视觉问答：visual question answering(title)；分类 cs.CV

AI总结 MAVIS基准旨在评估多模态来源归因系统，通过多模态文档检索和长形式答案生成，揭示多模态设置下信息量、 groundedness 和流畅性之间的权衡与改进方向。

Comments AAAI 2026; code is available at https://github.com/seokwon99/MAVIS

2511.10591 2025-11-14 cs.CL cs.AI 74%

Mined Prompting and Metadata-Guided Generation for Wound Care Visual Question Answering

Bavana Durgapraveen, Sornaraj Sivasankaran, Abhinand Balachandran, Sriram Rajkumar

机构 * EXL Health AI Lab at MEDIQA-WV 2025（EXL健康AI实验室）

专题命中视觉问答：visual question answering(title)；分类 cs.AI

Comments 2 figures, 11 pages

2509.11986 2025-09-16 cs.CV cs.CL 74%

Lost in Embeddings: Information Loss in Vision-Language Models

Wenyan Li, Raphael Tang, Chengzu Li, Caiqi Zhang, Ivan Vulić, Anders Søgaard

机构 * University of Copenhagen（哥本哈根大学）； Microsoft（微软）； University of Cambridge（剑桥大学）

专题命中视觉问答：vision-language model(title)；分类 cs.CV

2505.00065 2025-05-02 cs.CL cs.LG 74%

ConSens: Assessing context grounding in open-book question answering

Ivan Vankov, Matyo Ivanov, Adriana Correia, Victor Botev

机构 * Iris.ai BG

专题命中视觉问答：grounding(title)；分类 cs.LG

Comments 9 pages, 3 figures, 3 tables

2501.00750 2025-04-17 cs.AI 74%

Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform

Cheonsu Jeong

专题命中视觉问答：multimodal large language model(title)；分类 cs.AI

Comments 22 pages, 27 figures

Journal ref 2025 Journal of Intelligence and Information Systems

2412.10749 2024-12-17 cs.MM cs.CV 74%

Patch-level Sounding Object Tracking for Audio-Visual Question Answering

Zhangbin Li, Jinxing Zhou, Jing Zhang, Shengeng Tang, Kun Li, Dan Guo

专题命中视觉问答：visual question answering(title)；分类 cs.CV

Comments Accepted by AAAI 2025

2410.14132 2024-10-25 cs.CV cs.CL 74%

ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

Nghia Hieu Nguyen, Tho Thanh Quan, Ngan Luu-Thuy Nguyen

专题命中视觉问答：visual question answering(title)；分类 cs.CV

Comments PACLIC 2024

2405.10736 2024-05-20 cs.CV 74%

StackOverflowVQA: Stack Overflow Visual Question Answering Dataset

Motahhare Mirzaei, Mohammad Javad Pirhadi, Sauleh Eetemadi

专题命中视觉问答：visual question answering(title)；分类 cs.CV

2404.12720 2024-04-22 cs.CV cs.CL 74%

PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering

Yihao Ding, Kaixuan Ren, Jiabin Huang, Siwen Luo, Soyeon Caren Han

专题命中视觉问答：visual question answering(title)；分类 cs.CV

Comments Accepted by IJCAI 2024

2404.01151 2024-04-02 cs.CV 74%

Detect2Interact: Localizing Object Key Field in Visual Question Answering (VQA) with LLMs

Jialou Wang, Manli Zhu, Yulei Li, Honglei Li, Longzhi Yang, Wai Lok Woo

专题命中视觉问答：visual question answering(title)；分类 cs.CV

Comments Accepted to IEEE Intelligent Systems

2312.12222 2023-12-20 cs.CV 74%

EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answering

Junjue Wang, Zhuo Zheng, Zihang Chen, Ailong Ma, Yanfei Zhong

专题命中视觉问答：visual question answering(title)；分类 cs.CV

Comments Accepted By AAAI 2024

1. 视觉问答 3122 篇

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

RoboVQA: Multimodal Long-Horizon Reasoning for Robotics

Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning

Visually Grounded VQA by Lattice-based Retrieval

VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives

SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

Neural Reasoning, Fast and Slow, for Video Question Answering

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images

Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA

CrossView: Can Vision-Language Models Reason Across Cameras?

Binary Tracking for Spatial QA and Navigation with Open Vision-Language Models

A Comprehensive Survey of Knowledge-Based Vision Question Answering Systems: The Lifecycle of Knowledge in Visual Reasoning Task

Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusion

Can Vision-Language Models Answer Face to Face Questions in the Real-World?

IRIS: Intent Resolution via Inference-time Saccades for Open-Ended VQA in Large Vision-Language Models

Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs

Towards Understanding Best Practices for Quantization of Vision-Language Models

MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering

Mined Prompting and Metadata-Guided Generation for Wound Care Visual Question Answering

Lost in Embeddings: Information Loss in Vision-Language Models

ConSens: Assessing context grounding in open-book question answering

Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform

Patch-level Sounding Object Tracking for Audio-Visual Question Answering

ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

StackOverflowVQA: Stack Overflow Visual Question Answering Dataset

PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering

Detect2Interact: Localizing Object Key Field in Visual Question Answering (VQA) with LLMs

EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answering