视觉大模型 / VLM

Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLM

Lei Yu, Yechao Zhang, Ziqi Zhou, Yang Wu, Wei Wan, Minghui Li, Shengshan Hu, Pei Xiaobing, Jing Wang

专题命中 视觉问答 ：VLM(title,abstract)；vision-language model(abstract)；visual question answering(abstract)

Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Ugly

Lianghan Dong, Anamaria Crisan

专题命中 视觉问答 ：vision language model(title,abstract)；VLM(abstract)；LLaVA(abstract)

Advancing Egocentric Video Question Answering with Multimodal Large Language Models

Alkesh Patel, Vibhav Chitalia, Yinfei Yang

专题命中 视觉问答 ：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 8 pages

UniRAG: Universal Retrieval Augmentation for Large Vision Language Models

Sahel Sharifymoghaddam, Shivani Upadhyay, Wenhu Chen, Jimmy Lin

专题命中 视觉问答 ：vision language model(title,abstract)；LLaVA(abstract)；visual question answering(abstract)

Comments 14 pages, 6 figures

Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types

Neelabh Sinha, Vinija Jain, Aman Chadha

专题命中 视觉问答 ：vision-language model(title,abstract)；InternVL(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at The First Workshop of Evaluation of Multi-Modal Generation (EvalMG) in 31st International Conference on Computational Linguistics (COLING), 2025. 8 pages + references + 6 pages of Appendix

Effectiveness Assessment of Recent Large Vision-Language Models

Yao Jiang, Xinyu Yan, Ge-Peng Ji, Keren Fu, Meijun Sun, Huan Xiong, Deng-Ping Fan, Fahad Shahbaz Khan

专题命中 视觉问答 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by Visual Intelligence

Journal ref Visual Intelligence, 2024, Vol. 2, article no. 17

Vision-Language Model Based Handwriting Verification

Mihir Chauhan, Abhishek Satbhai, Mohammad Abuzar Hashemi, Mir Basheer Ali, Bina Ramamurthy, Mingchen Gao, Siwei Lyu, Sargur Srihari

专题命中 视觉问答 ：vision-language model(title)；vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 4 Pages, 1 Figure, 1 Table, Accepted as Short paper at Irish Machine Vision and Image Processing (IMVIP) Conference

Vision-Language Models as Success Detectors

Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, Serkan Cabi

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Flamingo: a Visual Language Model for Few-Shot Learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, Karen Simonyan

专题命中 视觉问答 ：visual language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 54 pages. In Proceedings of Neural Information Processing Systems (NeurIPS) 2022

'Just because you are right, doesn't mean I am wrong': Overcoming a Bottleneck in the Development and Evaluation of Open-Ended Visual Question Answering (VQA) Tasks

Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；visual reasoning(abstract)

Comments accepted to EACL 2021

What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning

Jae Hee Lee, Matthias Kerzel, Kyra Ahrens, Cornelius Weber, Stefan Wermter

专题命中 视觉问答 ：grounding(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to IJCAI 2022

GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering

Drew A. Hudson, Christopher D. Manning

专题命中 视觉问答 ：visual reasoning(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published as a conference paper at CVPR 2019 (oral)

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

Aishwarya Agrawal, Dhruv Batra, Devi Parikh, Aniruddha Kembhavi

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 15 pages, 10 figures. To appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions

Akash Ghosh, Arkadeep Acharya, Sriparna Saha, Vinija Jain, Aman Chadha

机构 * Department of Computer Science and Engineering, IIT Patna, India（印度帕纳大学计算机科学与工程系） ； Stanford University（斯坦福大学） ； Amazon AI（亚马逊人工智能） ； Indian Institute of Technology Patna, India（印度帕纳大学） 纠错

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI；visual language model(comments)

Comments One of the first survey on Visual Language Models

GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models

Guanxi Shen

机构 * Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Keywords: Explainable Computer Vision, Large Vision-Language Models, AI Interpretability, Explainable AI, Visual Saliency, Attribution Maps, Cross-Modal Attribution, Human Attention Alignment, AI Transparency

Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization

Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI；VLM(comments)

Comments Accepted at Advanced Robotics, website - https://haraduka.github.io/vlm-bbo/

Dynamic Clue Bottlenecks: Towards Interpretable-by-Design Visual Question Answering

Xingyu Fu, Ben Zhou, Sihao Chen, Mark Yatskar, Dan Roth

专题命中 视觉问答 ：visual question answering(title,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Multimodal, Visual Question Answering, Vision and Language

LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models

Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, Ping Luo

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 28 pages, 10 figures, a comprehensive evaluation of large vision-language models

MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents

MetaSpace：面向具身智能体空间认知的变形测试

Gengyang Xu, Dongwei Xiao, Yiteng Peng, Shuai Wang

机构 * Hong Kong University of Science and Technology（香港科技大学） 纠错

专题命中 视觉问答 ：MLLM(summary_cn,abstract)；visual question answering(abstract)；分类 cs.AI

AI总结 MetaSpace是评估具身智能体空间认知的框架，基于变形测试原则生成测试用例，在三个场景中检测到SOTA MLLM驱动智能体的90422个空间认知错误，其SC分数远低于人类基准。

Comments 30 pages, 17 figures. Published in Proceedings of the ACM on Programming Languages (OOPSLA1)

Journal ref Proceedings of the ACM on Programming Languages, 10, OOPSLA1 (April 2026), 343-372

Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization

提示优化器是盲目运行的吗？用于自动提示优化的跨模态视觉反馈

Haoyue Liu, Xiaoyu Ma, Ye Chen, Yuexian Zou, Xiaoying Tang

专题命中 视觉问答 ：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.AI

AI总结 研究多模态任务中自动提示优化的盲目性问题，提出跨模态视觉反馈方法，含故障条件视觉诊断和错误感知聚合阶段，在多个VQA数据集和目标VLM上效果显著，能提升分数且优化器可跨模型转移。

Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain

医学VQA中通过置信度-证据贝叶斯增益实现确定性幻觉检测

Mohammad Asadi, Tahoura Nedaee, Jack W. O'Sullivan, Euan Ashley, Ehsan Adeli

机构 * Department of Electrical Engineering, Stanford University, CA, USA（电气工程系，斯坦福大学） ； Department of Biology, Stanford University, CA, USA（生物学系，斯坦福大学） ； Division of Cardiology, Department of Medicine, Stanford University, CA, USA（心脏病学部，医学系，斯坦福大学） ； Department of Biomedical Data Science, Stanford University, CA, USA（生物医学数据科学系，斯坦福大学） ； Department of Computer Science, Stanford University, CA, USA（计算机科学系，斯坦福大学） ； Department of Psychiatry and Behavioral Sciences, Stanford University, CA, USA（精神病学与行为科学系，斯坦福大学） 纠错

专题命中 视觉问答 ：MLLM(summary_cn,abstract_cn)；visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.AI

AI总结 本文提出CEBaG方法，利用模型自身log概率中的不一致置信度和弱视觉证据敏感性，实现无需随机采样和外部模型的确定性幻觉检测，在医疗MLLM和VQA基准测试中取得最佳AUC表现。

FlameVQA: A Physically-Grounded UAV Wildfire VQA Benchmark with Radiometric Thermal Supervision

FlameVQA: 一个基于辐射热监督的物理基础无人机野火VQA基准

Mobin Habibpour, John Spodnik, Niloufar Alipour Talemi, Fatemeh Afghah

机构 * National Science Foundation（国家科学基金会） ； NASA（美国国家航空航天局） 纠错

专题命中 视觉问答 ：MLLM(summary_cn,abstract)；visual question answering(abstract)；分类 cs.CV

AI总结 提出FlameVQA，一个基于FLAME 3数据集的无人机野火多选视觉问答基准，利用配对的RGB和辐射热TIFF图像实现温度基础的推理，评估了多种MLLM在检测、定位、覆盖估计等任务上的表现。

UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoning

UltraVR：面向证据推理的诊断性超分辨率图像VQA基准

Gexin Huang, Yanting Yang, Myeongkyun Kang, Beidi Zhao, Jun Zhou, Chen Zhou, Gang Wang, Zu-hua Gao, Xiaoxiao Li

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Vector Institute（向量研究所） ； BC Cancer Agency（不列颠哥伦比亚癌症中心） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract_cn)；visual reasoning(abstract)；visual question answering(abstract)

AI总结 提出UltraVR基准，通过结构化思维链标注诊断视觉语言模型在超分辨率图像上的证据推理能力，发现模型在证据定位和局部感知环节错误集中。

Comments 10 pages, 1 figure

Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams

Enginuity：工程图纸视觉语言理解的数据集与基准

Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal

机构 * Predii ； Oak Ridge National Laboratory（橡树岭国家实验室） ； Independent Researcher（独立研究员） 纠错

专题命中 视觉问答 ：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV

AI总结 针对工程图纸领域缺乏公开基准的问题，提出首个开放数据集Enginuity，通过结构化零件表提取和自由形式视觉问答两项任务评估前沿VLM，揭示零件识别与描述保真度之间的系统性差距。

Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision

面向低视力人群的基于VQA的事件地图城市风险感知导航

Antoni Valls, Jordi Sanchez-Riera

机构 * Institut de Robòtica i Informàtica Industrial, CSIC-UPC（机器人与信息技术研究所，CSIC-UPC） 纠错

专题命中 视觉问答 ：LLaVA(abstract,abstract_cn)；vision-language model(abstract)；visual question answering(abstract)；multimodal large language model(abstract)

AI总结 本文提出基于视觉问答的事件地图框架，利用视觉语言模型进行行人场景描述和危险识别，通过三级查询结构实现细粒度场景理解，生成风险感知导航地图，验证多模态大语言模型在助视导航中的有效性。

Comments 10 pages, 6 figures, submitted to IEEE T-ITS

ATIR: Towards Audio-Text Interleaved Contextual Retrieval

ATIR：面向音频-文本交错上下文检索

Tong Zhao, Chenghao Zhang, Yutao Zhu, Zhicheng Dou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院） 纠错

专题命中 视觉问答 ：MLLM(summary_cn,abstract)；multimodal large language model(abstract)；分类 cs.AI

AI总结 本文提出ATIR任务，通过整合ASR、QA和检索数据集构建基准，解决现有音频检索数据集在语义检索上的局限，引入新的token压缩机制提升MLLM在ATIR中的性能。

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

二次审视：多模态大语言模型中的免训练证据高亮

Marco Morini, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

机构 * University of Modena and Reggio Emilia（摩德纳和雷焦艾米利亚大学） 纠错

专题命中 视觉问答 ：visual question answering(title,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出Look Twice框架，通过模型注意力模式识别相关视觉区域和文本证据，提升预训练MLLMs在多模态证据利用中的表现，实验显示在多个VQA基准上效果显著。

Comments Project Page: https://aimagelab.github.io/LoT/

SPARC-Rad: A Multimodal Benchmark Dataset and Evaluation Pipeline for Spatial and Anatomical Reasoning in Radiology Vision-Language Models

SPARC-Rad：面向放射学视觉语言模型的空间与解剖推理多模态基准数据集及评估流程

Satvik Tripathi, Mustafa Ege Seker, Kristian Quevada, Ebubechukwu D Enwerem, Pratham Khandelwal, Emine Meltem, Bera Koca, Shahriar Faghani, Jacinta Arnold, Dania Daye, Tessa S. Cook

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

AI总结 该研究开发了SPARC-Rad多模态基准数据集及评估流程，用于评估放射学VLMs的空间与解剖推理能力，为相关模型的开发与部署前评估提供支持。

QR-Structured Thermal Triggers for Targeted Semantic Attacks on Infrared Vision-Language Models

用于红外视觉语言模型定向语义攻击的QR结构化热触发装置

Xiang Chen, Yingying Zhao, Chao Li, Jiaju Han, Ben Zhang, Ang Li, Jiahuan Long, Yiwei Wei, Jiujiang Guo, Chengyin Hu

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

AI总结 本研究提出QR-STT框架，作为隐蔽黑盒攻击手段，可定向操控IR-VLMs的语义对齐，其扰动还能跨任务迁移，凸显了对该类攻击进行鲁棒性评估的必要性。

An Empirical Analysis of Continual Learning for Heterogeneous Medical Visual Question Answering

异构医学视觉问答持续学习的实证分析

Mai A. Shaaban, Tausifa Jan Saleem, Alaa Mohamed, Dilnaz Utemissova, Ufaq Khan, Mohammad Yaqub

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； Alexandria University（亚历山大大学） 纠错

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 研究异构医学视觉问答持续学习，通过系统评估多种临床目标任务，探究现有CL方法减轻灾难性遗忘能力、对任务排序敏感性及低秩适应参数演变，发现交错不同任务时现有方法难维持稳定性 - 可塑性平衡。