视觉大模型 / VLM - arXivDaily 专题

2512.04763 2025-12-05 cs.LG cs.CL cs.CV 82%

MemLoRA: Distilling Expert Adapters for On-Device Memory Systems

MemLoRA: 通过专家适配器实现设备端记忆系统

Massimo Bini, Ondrej Bohdal, Umberto Michieli, Zeynep Akata, Mete Ozay, Taha Ceritli

机构 * Samsung R&D Institute UK（三星英国研发中心）； Technical University of Munich（慕尼黑技术大学）； Helmholtz Munich（慕尼黑海德堡研究所）

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；visual reasoning(abstract)；visual question answering(abstract)

AI总结 MemLoRA通过为小型语言模型添加专用记忆适配器，实现设备端记忆系统的本地部署，并扩展至视觉理解任务，提升多模态场景下的性能。

2511.11034 2025-11-17 cs.CV cs.AI 82%

CrossMed: A Multimodal Cross-Task Benchmark for Compositional Generalization in Medical Imaging

Pooja Singh, Siddhant Ujjain, Tapan Kumar Gandhi, Sandeep Kumar

专题命中视觉问答：vision-language model(abstract)；LLaVA(abstract)；visual question answering(abstract)；multimodal large language model(abstract)

2510.22728 2025-10-28 cs.LG cs.CV 82%

S-Chain: Structured Visual Chain-of-Thought For Medicine

Khai Le-Duc, Duy M. H. Nguyen, Phuong T. H. Trinh, Tien-Phat Nguyen, Nghiem T. Diep, An Ngo, Tung Vu, Trinh Vuong, Anh-Tien Nguyen, Mau Nguyen, Van Trung Hoang, Khai-Nguyen Nguyen, Hy Nguyen, Chris Ngo, Anji Liu, Nhat Ho, Anne-Christin Hauschild, Khanh Xuan Nguyen, Thanh Nguyen-Tang, Pengtao Xie, Daniel Sonntag, James Zou, Mathias Niepert, Anh Totti Nguyen

专题命中视觉问答：vision-language model(abstract)；LLaVA(abstract)；visual question answering(abstract)；grounding(abstract)

Comments First version

2508.18297 2025-08-27 cs.CV cs.AI cs.CL 82%

Can VLMs Recall Factual Associations From Visual References?

Dhananjay Ashok, Ashutosh Chaubey, Hirona J. Arai, Jonathan May, Jesse Thomason

机构 * University of Southern California（南加州大学）； Information Sciences Institute, University of Southern California（信息科学研究所，南加州大学）

专题命中视觉问答：vision language model(abstract)；VLM(abstract)；visual question answering(abstract)；grounding(abstract)

Comments To appear at EMNLP 2025 (Findings)

2403.15585 2025-04-17 cs.CV cs.AI 82%

MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis

Mai A. Shaaban, Adnan Khan, Mohammad Yaqub

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；multimodal large language model(abstract)；MLLM(abstract)

2411.15556 2025-03-31 cs.CV cs.AI 82%

ReWind: Understanding Long Videos with Instructed Learnable Memory

Anxhelo Diko, Tinghuai Wang, Wassim Swaileh, Shiyan Sun, Ioannis Patras

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)；grounding(abstract)

2412.10151 2024-12-16 cs.CV cs.AI cs.CL 82%

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation

Hyeonseok Lim, Dongjae Shin, Seohyun Song, Inho Won, Minjun Kim, Junghun Yuk, Haneol Jang, KyungTae Lim

专题命中视觉问答：vision language model(abstract)；VLM(abstract)；LLaVA(abstract)；visual question answering(abstract)

Comments The 31st International Conference on Computational Linguistics (COLING 2025), 19 pages

2403.08773 2024-10-29 cs.CV cs.AI cs.CL cs.MM 82%

Veagle: Advancements in Multimodal Representation Learning

Rajat Chawla, Arkajit Datta, Tushar Verma, Adarsh Jha, Anmol Gautam, Ayush Vatsal, Sukrit Chaterjee, Mukunda NS, Ishaan Bhola

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；grounding(abstract)；multimodal large language model(abstract)

2409.20424 2024-10-01 cs.CV cs.AI 82%

World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering

Jiacong Wang, Bohong Wu, Haiyong Jiang, Xun Zhou, Xin Xiao, Haoyuan Guo, Jun Xiao

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)；grounding(abstract)

Comments Accepted at EMNLP 2024 Main Conference, 16pages

2607.04610 2026-07-07 cs.RO 新提交 82%

RoboVista: Evaluating Vision Language Models for Diverse Robot Applications

RoboVista：评估用于各种机器人应用的视觉语言模型

Shuangyu Xie, Kaiyuan Chen, Ziyang Chen, Simeon Adebola, Yixuan Huang, Zehan Ma, Tianshuang Qiu, Wentao Yuan, Dhruv Shah, Pannag R. Sanketi, Ken Goldberg

机构 * University of California, Berkeley（加州大学伯克利分校）； Princeton University（普林斯顿大学）； Google DeepMind（谷歌DeepMind）

专题命中视觉问答：vision language model(title)；vision-language model(abstract)；visual question answering(abstract)

AI总结研究针对机器人多样应用，提出模块化评估框架Robot Question Answering及基准RoboVista，其源于真实机器人系统等，含多任务类型VQA实例，实验表明现有视觉语言模型有差距，且与现实任务执行相关。

Comments Accepted to RSS 2026. Project website: https://berkeleyautomation.github.io/robovista/

2604.11632 2026-05-26 cs.CL 82%

CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

CArtBench: 评估视觉语言模型在中国艺术理解、解读与真实性方面的能力

Xuefeng Wei, Zhixuan Wang, Xuan Zhou, Zhi Qu, Hongyao Li, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe

机构 * Nara Institute of Science and Technology（奈良科学技術大學）； Liaoning Normal University（遼寧師範大學）

专题命中视觉问答：vision-language model(title,abstract)；VLM(abstract_cn)

AI总结提出CARTBENCH基准，通过四个子任务评估视觉语言模型在中国艺术作品上的识别、推理、鉴赏和真伪鉴别能力，发现现有模型在复杂证据链接、风格断代和真伪诊断方面表现不足。

Comments under review

2412.13050 2026-05-13 cs.LG cs.AI cs.CL cs.CV cs.SD eess.AS 82%

Modality-Inconsistent Continual Learning of Multimodal Large Language Models

多模态大语言模型的模态不一致持续学习

Weiguo Pian, Shijian Deng, Shentong Mo, Mingrui Liu, Yunhui Guo, Yapeng Tian

机构 * The University of Texas at Dallas（德克萨斯大学达拉斯分校）； Carnegie Mellon University（卡内基梅隆大学）； George Mason University（乔治·梅森大学）

专题命中视觉问答：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出MICL，一种针对多模态大语言模型的持续学习场景，涉及不一致的模态和变化的任务类型。通过伪目标生成模块和基于指令的知识蒸馏，解决任务类型转移导致的遗忘问题，实验验证了方法的有效性。

Comments Accepted at Transactions on Machine Learning Research (TMLR), 2026

2604.01259 2026-04-03 cs.RO 82%

Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models

Bench2Drive-VL: 用于基于视觉语言模型的闭环自动驾驶的基准测试

Xiaosong Jia, Yuqian Shao, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, Junchi Yan

机构 * Shanghai Key Laboratory of Multimodal Embodied AI（上海市多模态具身智能重点实验室）

专题命中视觉问答：vision-language model(title,abstract)；VLM(abstract)

AI总结本文提出Bench2Drive-VL，通过闭环评估方法提升视觉语言模型在自动驾驶中的性能，包含DriveCommenter生成多样化问题对、统一协议接口、灵活推理框架及完整开发生态，开源代码和标注数据。

Comments All codes and annotated datasets are available at \url{https://github.com/Thinklab-SJTU/Bench2Drive-VL} and \url{https://huggingface.co/datasets/Telkwevr/Bench2Drive-VL-base}

2604.01002 2026-04-02 cs.CV cs.AI cs.LG 82%

Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding

基于证据的关键帧采样用于基于MLLM的长视频理解

Yiheng Wang, Lichen Zhu, Yueqian Lin, Yudong Liu, Jingyang Zhang, Hai "Helen" Li, Yiran Chen

机构 * Duke University（杜克大学）； Independent Researcher（独立研究员）

专题命中视觉问答：MLLM(title)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出基于信息瓶颈理论的证据驱动关键帧采样框架，通过最大化选帧与查询的条件互信息，提升长视频理解性能，实验表明在严格token预算下优于现有方法。

2603.16092 2026-03-18 cs.CV cs.AI cs.LG 82%

Parallel In-context Learning for Large Vision Language Models

大规模视觉语言模型的并行上下文学习

Shin'ya Yamaguchi, Daiki Chijiwa, Tamao Sakao, Taku Hasegawa

机构 * NTT（日本电信电话研究所）

专题命中视觉问答：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出并行上下文学习方法，通过将长上下文分割为短片段并行处理，提升大视觉语言模型的推理效率，同时保持性能与传统多模态上下文学习相当。

Comments Accepted to CVPR 2026 (Findings); Code is available at https://github.com/yshinya6/parallel-icl

2509.21991 2026-03-18 cs.CV cs.AI cs.CL cs.LG 82%

ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models

ERGO：高效高分辨率视觉理解用于视觉-语言模型

Jewon Lee, Wooksu Shin, Seungmin Yang, Ki-Ung Song, DongUk Lim, Jaeyeon Kim, Tae-Ho Kim, Bo-Kyeong Kim

机构 * Nota Inc.（Nota公司）

专题命中视觉问答：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 ERGO提出一种两阶段'粗到细'推理流程，通过下采样图像识别任务相关区域，再以全分辨率裁剪处理，从而在降低计算成本的同时保留必要的细粒度视觉细节，提升视觉-语言模型的效率与准确性。

2411.08253 2026-03-12 cs.RO 82%

Open-World Task and Motion Planning via Vision-Language Model Generated Constraints

开放世界任务与运动规划 via 视觉-语言模型生成的约束

Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Caelan Reed Garrett

机构 * NVIDIA Research（NVIDIA研究）； MIT CSAIL

专题命中视觉问答：vision-language model(title,abstract)；visual question answering(abstract)

AI总结本文提出OWL-TAMP方法，通过整合视觉-语言模型生成的约束，提升开放世界任务与运动规划的性能，实现对自然语言目标的处理能力。

Comments A version of this paper appears in IEEE Robotics and Automation Letters (RA-L) Volume 11, Issue 3

2506.06905 2026-03-03 cs.AI cs.CL cs.CV cs.LG 82%

Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering

元适应性提示蒸馏用于少样本视觉问答

Akash Gupta, Amos Storkey, Mirella Lapata

专题命中视觉问答：visual question answering(title,abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出元适应性提示蒸馏方法，通过固定软提示提升少样本视觉问答性能，实验表明在低数据情况下优于ICL和参数高效微调方法。

Comments ICLR 2026

2602.02185 2026-03-03 cs.CV cs.AI cs.CL cs.LG 82%

Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models

视觉-深度研究基准：重新思考多模态大语言模型的视觉和文本搜索

Yu Zeng, Wenxuan Huang, Zhen Fang, Shuang Chen, Yufan Shen, Yishuo Cai, Xiaoman Wang, Zhenfei Yin, Lin Chen, Zehui Chen, Shiting Huang, Yiming Zhao, Xu Tang, Yao Hu, Philip Torr, Wanli Ouyang, Shaosheng Cao

专题命中视觉问答：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出Vision-DeepResearch基准，通过精心设计的问题评估多模态大语言模型的视觉与文本搜索能力，并引入多轮裁剪搜索流程提升实际检索性能。

2509.18053 2026-02-17 cs.RO 82%

V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts

V2V-GoT: 基于多模态大语言模型和思维图的车对车协同自动驾驶

Hsu-kuang Chiu, Ryo Hachiuma, Chien-Yi Wang, Yu-Chiang Frank Wang, Min-Hung Chen, Stephen F. Smith

机构 * NVIDIA ； Carnegie Mellon University（卡内基梅隆大学）

专题命中视觉问答：multimodal large language model(title,abstract)；MLLM(abstract)

AI总结本文提出V2V-GoT框架，结合多模态大语言模型和图-思维方法，提升车对车协同自动驾驶的感知、预测和规划能力。

Comments Accepted by ICRA 2026 (IEEE International Conference on Robotics and Automation). Project: https://eddyhkchiu.github.io/v2vgot.github.io/ Code: https://github.com/eddyhkchiu/V2V-GoT Dataset: https://huggingface.co/datasets/eddyhkchiu/V2V-GoT-QA

2511.03328 2025-11-06 cs.CL cs.AI cs.CV cs.LG 82%

Benchmarking the Thinking Mode of Multimodal Large Language Models in Clinical Tasks

Jindong Hong, Tianjie Chen, Lingjie Luo, Chuanyang Zheng, Ting Xu, Haibao Yu, Jianing Qiu, Qianzhong Chen, Suning Huang, Yan Xu, Yong Gui, Yijun He, Jiankai Sun

机构 * Bytedance（字节跳动）； Peking University（北京大学）； The Chinese University of Hong Kong（香港中文大学）； The University of Hong Kong（香港大学）； Mohamed bin Zayed University of Artificial Intelligence（马尔代夫人工智能大学）； Stanford University（斯坦福大学）； University of Michigan（密歇根大学）

专题命中视觉问答：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2505.23759 2025-09-18 cs.CL cs.AI cs.CV cs.LG 82%

Puzzled by Puzzles: When Vision-Language Models Can't Take a Hint

Heekyung Lee, Jiaxin Ge, Tsung-Han Wu, Minwoo Kang, Trevor Darrell, David M. Chan

机构 * POSTECH ； University of California, Berkeley（加州大学伯克利分校）

专题命中视觉问答：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2025 Main Conference

2505.19455 2025-09-12 cs.CV cs.AI cs.LG 82%

MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering

Xu Li, Fan Lyu

机构 * Khoury College of Computer Sciences, Northeastern University（东北大学克劳尔计算机科学学院）； New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）

专题命中视觉问答：visual question answering(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2505.07084 2025-09-12 cs.RO 82%

DriveSOTIF: Advancing Perception SOTIF Through Multimodal Large Language Models

Shucheng Huang, Freda Shi, Chen Sun, Jiaming Zhong, Minghao Ning, Yufeng Yang, Yukun Lu, Hong Wang, Amir Khajepour

机构 * MVSLab, Department of Mechanical and Mechatronics Engineering, University of Waterloo（滑铁卢大学机械与机电工程系MVSLab）； CompLING Lab, David R. Cheriton School of Computer Science, University of Waterloo（滑铁卢大学大卫·R·切里顿计算机科学学院CompLING Lab）； Department of Data and Systems Engineering, University of Hong Kong（香港大学数据与系统工程系）； Department of Mechanical Engineering, University of New Brunswick（新不伦瑞克大学机械工程系）； School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动性学院）

专题命中视觉问答：multimodal large language model(title,abstract)；MLLM(abstract)

Comments This work has been accepted to IEEE Transactions on Vehicular Technology. Please refer to the copyright notice for additional information

2507.02357 2025-07-04 cs.CL 82%

Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models

Christian Jaumann, Annemarie Friedrich, Rainer Lienhart

机构 * University of Augsburg（奥格斯堡大学）

专题命中视觉问答：multimodal large language model(title,abstract)；visual question answering(abstract)

Comments Accepted at 5th Workshop on Scholarly Document Processing @ ACL 2025

2505.21755 2025-06-24 cs.CV cs.AI cs.CL cs.LG 82%

FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering

Chengyue Huang, Brisa Maneechotesuwan, Shivang Chopra, Zsolt Kira

机构 * Georgia Institute of Technology（佐治亚理工学院）

专题命中视觉问答：visual question answering(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2025

2503.18491 2025-06-12 cs.CL 82%

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

Shuo Yang, Siwen Luo, Soyeon Caren Han, Eduard Hovy

机构 * The University of Melbourne（墨尔本大学）； The University of Western Australia（西澳大学）

专题命中视觉问答：visual question answering(title,abstract)；vision-language model(abstract)

Comments Findings of ACL 2025

2412.06141 2025-06-05 cs.CV cs.AI cs.CL cs.LG 82%

MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization

Kangyu Zhu, Peng Xia, Yun Li, Hongtu Zhu, Sheng Wang, Huaxiu Yao

机构 * UNC Chapel-Hill（UNC夏洛特-希尔分校）； Brown University（布朗大学）； University of Washington（华盛顿大学）

专题命中视觉问答：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2025

2503.03854 2025-06-02 cs.CL 82%

Vision-Language Models Struggle to Align Entities across Modalities

Iñigo Alonso, Gorka Azkune, Ander Salaberria, Jeremy Barnes, Oier Lopez de Lacalle

机构 * School of Informatics, The University of Edinburgh（信息学院，爱丁堡大学）； HiTZ Center - Ixa, University of the Basque Country UPV/EHU（巴斯克国家大学HiTZ中心）

专题命中视觉问答：vision-language model(title,abstract)；VLM(abstract)

Comments Accepted Findings ACL 2025

2406.11193 2025-05-21 cs.CL 82%

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））； The Hong Kong University of Science and Technology（香港科学与技术大学）； Tongji University（同济大学）

专题命中视觉问答：multimodal large language model(title,abstract)；visual question answering(abstract)

Comments Accepted by the Main Conference of Empirical Methods in Natural Language Processing (EMNLP) 2024

1. 视觉问答 3115 篇

MemLoRA: Distilling Expert Adapters for On-Device Memory Systems

CrossMed: A Multimodal Cross-Task Benchmark for Compositional Generalization in Medical Imaging

S-Chain: Structured Visual Chain-of-Thought For Medicine

Can VLMs Recall Factual Associations From Visual References?

MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis

ReWind: Understanding Long Videos with Instructed Learnable Memory

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation

Veagle: Advancements in Multimodal Representation Learning

World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering

RoboVista: Evaluating Vision Language Models for Diverse Robot Applications

CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

Modality-Inconsistent Continual Learning of Multimodal Large Language Models

Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models

Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding

Parallel In-context Learning for Large Vision Language Models

ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models

Open-World Task and Motion Planning via Vision-Language Model Generated Constraints

Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering

Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models

V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts

Benchmarking the Thinking Mode of Multimodal Large Language Models in Clinical Tasks

Puzzled by Puzzles: When Vision-Language Models Can't Take a Hint

MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering

DriveSOTIF: Advancing Perception SOTIF Through Multimodal Large Language Models

Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models

FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization

Vision-Language Models Struggle to Align Entities across Modalities

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model