视觉大模型 / VLM

A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

Zhe Xu, Ziyi Liu, Junlin Hou, Jiabo Ma, Cheng Jin, Yihui Wang, Zhixuan Chen, Zhengyu Zhang, Fuxiang Huang, Zhengrui Guo, Fengtao Zhou, Yingxue Xu, Xi Wang, Ronald Cheong Kin Chan, Li Liang, Hao Chen

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Southern Medical University（南方医科大学） ； Nanfang Hospital and School of Basic Medical Sciences（南方医院和基础医学科学学院） ； Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 视觉问答 ：multimodal large language model(title,abstract)；visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets

Kaiyuan Chen, Shuangyu Xie, Zehan Ma, Pannag R Sanketi, Ken Goldberg

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis

Robinson Umeike, Neil Getty, Fangfang Xia, Rick Stevens

专题命中 视觉问答 ：vision-language model(title)；vision language model(abstract)；LLaVA(abstract)；visual question answering(abstract)

Comments 4 Pages, 4 Figures, 1 Table

Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving

Ming Liu, Siyuan Liang, Koushik Howlader, Liwen Wang, Dacheng Tao, Wensheng Zhang

机构 * Iowa State University（爱荷华州立大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 视觉问答 ：vision language model(title)；vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)

Data Metabolism: An Efficient Data Design Schema For Vision Language Model

Jingyuan Zhang, Hongzhi Zhang, Zhou Haonan, Chenxi Sun, Xingguang ji, Jiakang Wang, Fanheng Kong, Yahui Liu, Qi Wang, Fuzheng Zhang

专题命中 视觉问答 ：vision language model(title)；VLM(abstract)；visual language model(abstract)；visual question answering(abstract)

Comments To be presented at ICLR 2025, First Workshop on Open Science for Foundation Models

Augmenting Multimodal LLMs with Self-Reflective Tokens for Knowledge-based Visual Question Answering

Federico Cocchi, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中 视觉问答 ：visual question answering(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language

Yoonshik Kim, Jaeyoon Jung

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPRW 2025, Workshop on Benchmarking and Expanding AI Multimodal Approaches

LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression

Souvik Kundu, Anahita Bhiwandiwalla, Sungduk Yu, Phillip Howard, Tiep Le, Sharath Nittur Sridhar, David Cobbley, Hao Kang, Vasudev Lal

专题命中 视觉问答 ：vision-language model(title,abstract)；LLaVA(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments This work has been accepted to NAACL 2025

Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learning

Rui Zhao, Qirui Yuan, Jinyu Li, Haofeng Hu, Yun Li, Chengyuan Zheng, Fei Gao

专题命中 视觉问答 ：MLLM(title,abstract)；visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

PitVQA++: Vector Matrix-Low-Rank Adaptation for Open-Ended Visual Question Answering in Pituitary Surgery

Runlong He, Danyal Z. Khan, Evangelos B. Mazomenos, Hani J. Marcus, Danail Stoyanov, Matthew J. Clarkson, Mobarakol Islam

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 9 pages

Efficient Few-Shot Continual Learning in Vision-Language Models

Aristeidis Panos, Rahaf Aljundi, Daniel Olmeda Reino, Richard E. Turner

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Learning to Compress Contexts for Efficient Knowledge-based Visual Question Answering

Weixi Weng, Jieming Zhu, Xiaojun Meng, Hao Zhang, Rui Zhang, Chun Yuan

专题命中 视觉问答 ：visual question answering(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, Chong Ruan

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis

Ferhat Ozgur Catak, Murat Kuzlu, Taylor Patrick

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 15 pages

LIME: Less Is More for MLLM Evaluation

King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shawn Gavin, Tuney Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Zachary Liu, Xiang Yue, J. H. Liu, Chenghua Lin, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang

专题命中 视觉问答 ：MLLM(title,abstract)；visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering

Xinyue Hu, Lin Gu, Qiyuan An, Mengliang Zhang, Liangchen Liu, Kazuma Kobayashi, Tatsuya Harada, Ronald M. Summers, Yingying Zhu

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；vision language model(abstract)；分类 cs.CV、cs.LG

Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA

Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Xinze Guan, Xin Eric Wang

专题命中 视觉问答 ：multimodal large language model(title,abstract)；visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments ACL 2024

Advancing High Resolution Vision-Language Models in Biomedicine

Zekai Chen, Arda Pekis, Kevin Brown

专题命中 视觉问答 ：vision-language model(title,abstract)；LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 15 pages

FreeVA: Offline MLLM as Training-Free Video Assistant

Wenhao Wu

专题命中 视觉问答 ：MLLM(title,abstract)；LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Preprint. Work in progress

Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving

Akshay Gopalkrishnan, Ross Greer, Mohan Trivedi

专题命中 视觉问答 ：vision-language model(title,abstract)；vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 3 figures, Accepted at CVPR 2024 Vision and Language for Autonomous Driving and Robotics Workshop

VDC: Versatile Data Cleanser based on Visual-Linguistic Inconsistency by Multimodal Large Language Models

Zihao Zhu, Mingda Zhang, Shaokui Wei, Bingzhe Wu, Baoyuan Wu

专题命中 视觉问答 ：multimodal large language model(title,abstract)；visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2024

Physically Grounded Vision-Language Models for Robotic Manipulation

Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, Dorsa Sadigh

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Updated version for ICRA 2024

Uncertainty-Aware Evaluation for Vision-Language Models

Vasily Kostumov, Bulat Nutfullin, Oleg Pilipenko, Eugene Ilyushin

专题命中 视觉问答 ：vision-language model(title,abstract)；LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Detecting and Preventing Hallucinations in Large Vision Language Models

Anisha Gunjal, Jihan Yin, Erhan Bas

专题命中 视觉问答 ：vision language model(title,abstract)；LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments AAAI 2024

GeoChat: Grounded Large Vision-Language Model for Remote Sensing

Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, Fahad Shahbaz Khan

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 4 figures

Weakly Supervised Relative Spatial Reasoning for Visual Question Answering

Pratyay Banerjee, Tejas Gokhale, Yezhou Yang, Chitta Baral

专题命中 视觉问答 ：visual question answering(title,abstract)；visual reasoning(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ICCV 2021. PaperId : ICCV2021-10857 Copyright transferred to IEEE ICCV. DOI will be updated later

Prompt-Guided Prefiltering for VLM Image Compression

基于提示的VLM图像压缩预过滤

Bardia Azizian, Ivan V. Bajic

机构 * School of Engineering Science, Simon Fraser University（西蒙菲莎大学工程科学学院） 纠错

专题命中 视觉问答 ：VLM(title,abstract)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.AI

AI总结 本文提出一种轻量级、即插即用的提示引导预过滤模块，用于识别图像中与文本提示最相关的区域，从而提升VLM图像压缩效率，实验显示在保持任务准确性的同时实现25-50%的平均比特率降低。

Comments 7 pages, 5 figures. Accepted to IEEE ICME 2026. Code: https://github.com/bardia-az/pgp-vlm-compression

PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning

PhysMRV：用于物理合理性推理的物理内存检索与验证

Wenyuan Wang, Lianyu Hu, Hao Wang, Yang Liu

专题命中 视觉问答 ：VLM(summary_cn,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 针对视频语言模型物理合理性推理不可靠的问题，提出免训练的PhysMRV框架，通过将训练视频转化为分层内存库，利用结构化物理证据指导VLM验证物理合理性，在多基准测试中取得一致改进，有效增强该推理能力。

Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding

Qwen-3D：用于空间理解的通用三维视觉语言模型

Lucy Lin, Ayush Jain, Yifan Liu, Katerina Fragkiadaki

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 视觉问答 ：vision-language model(title)；visual reasoning(abstract)；visual question answering(abstract)；grounding(abstract)

AI总结 本研究提出 Qwen-3D 三维视觉语言模型，通过多视角几何线索与三维旋转位置嵌入提升空间推理，在三维任务上超越现有 3D LMM，还保持二维任务性能。

Comments Project Page: https://qwen-3d.github.io/

Failing to See or Failing to Know? Attributing Errors in Vision-Language Models

是看不见还是不知道？归因视觉语言模型中的错误

Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov, Timothy Baldwin, Yova Kementchedjhieva

机构 * MBZUAI The University of Melbourne（MBZUAI墨尔本大学） 纠错

专题命中 视觉问答 ：vision-language model(title,abstract)；VLM(abstract_cn)；visual question answering(abstract)；分类 cs.CV

AI总结 研究视觉语言模型在回答需额外知识问题时的错误，提出统一框架分离失败模式，探讨预生成信号能否预测错误源，发现可在解码前预测，能据此进行针对性干预。