视觉大模型 / VLM

EfficientEQA: An Efficient Approach to Open-Vocabulary Embodied Question Answering

Kai Cheng, Zhengyuan Li, Xingpeng Sun, Byung-Cheol Min, Amrit Singh Bedi, Aniket Bera

机构 * Purdue University（普渡大学） ； University of Central Florida（中央佛罗里达大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments IROS 2025 Oral

Visual Enumeration Remains Challenging for Multimodal Generative AI

Alberto Testolin, Kuinan Hou, Marco Zorzi

机构 * Department of General Psychology and Department of Mathematics University of Padova（帕多瓦大学心理学系和数学系） ； Department of General Psychology University of Padova（帕多瓦大学心理学系） ； Department of General Psychology and Padova Neuroscience Center University of Padova（帕多瓦大学心理学系和帕多瓦神经科学中心） ； IRCSS San Camillo Hospital, Venice-Lido（威尼斯利多医院IRCSS桑卡莫医院） 纠错

专题命中 视觉问答 ：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs

Chenchen Zhao, Zhengyuan Shi, Xiangyu Wen, Chengjie Liu, Yi Liu, Yunhao Zhou, Yuxiang Zhao, Hefei Feng, Yinan Zhu, Gwok-Waa Wan, Xin Cheng, Weiyu Chen, Yongqi Fu, Chujie Chen, Chenhao Xue, Guangyu Sun, Ying Wang, Yibo Lin, Jun Yang, Ning Xu, Xi Wang, Qiang Xu

机构 * Department of Computer Science and Engineering, The Chinese University of Hong Kong（中国香港中文大学计算机科学与工程系） ； School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院） ； School of Integrated Circuits, Peking University（北京大学集成电路学院） ； School of Intergrated Circuits, Southeast University（东南大学集成电路学院） ； School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院） ； Department of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术系） ； National Center of Technology Innovation for EDA（EDA技术创新国家中心） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 1 figure, 5 tables. To appear in ICCAD 2025

VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks

Juhwan Choi, Junehyoung Kwon, JungMin Yun, Seunguk Yu, YoungBin Kim

机构 * AITRICS Seoul（AITRICS首尔） ； Chung-Ang University（Chung-ang 大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2025 Workshop on Curated Data for Efficient Learning (CDEL)

Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractor

Vatsal Agarwal, Matthew Gwilliam, Gefen Kohavi, Eshan Verma, Daniel Ulbricht, Abhinav Shrivastava

机构 * University of Maryland（马里兰大学） ； Apple（苹果公司） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

Comments Website: see https://vatsalag99.github.io/mustafar/

HSENet: Hybrid Spatial Encoding Network for 3D Medical Vision-Language Understanding

Yanzhao Shi, Xiaodan Zhang, Junzhong Ji, Haoning Jiang, Chengxin Zheng, Yinong Wang, Liangqiong Qu

机构 * The University of Hong Kong（香港大学） ； Beijing University of Technology（北京工业大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 9 figures. arXiv admin note: text overlap with arXiv:2410.14200 by other authors

TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervision

Ayush Gupta, Anirban Roy, Rama Chellappa, Nathaniel D. Bastian, Alvaro Velasquez, Susmit Jha

机构 * SRI ； Johns Hopkins University（约翰霍普金斯大学） ； United States Military Academy（美国军事学院） ； University of Colorado Boulder（科罗拉多大学博尔德分校） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

TextSquare: Scaling up Text-Centric Visual Instruction Tuning

Jingqun Tang, Chunhui Lin, Zhen Zhao, Shu Wei, Binghong Wu, Qi Liu, Yangfan He, Kuan Lu, Hao Feng, Yang Li, Siqi Wang, Lei Liao, Wei Shi, Yuliang Liu, Hao Liu, Yuan Xie, Xiang Bai, Can Huang

机构 * ByteDance Inc.（字节跳动公司） ； East China Normal University（东华大学） ； Huazhong University of Science and Technology（华中科技大学） ； University of Minnesota（明尼苏达大学） ； Cornell University（康奈尔大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding

Feilong Tang, Chengzhi Liu, Zhongxing Xu, Ming Hu, Zelin Peng, Zhiwei Yang, Jionglong Su, Minquan Lin, Yifan Peng, Xuelian Cheng, Imran Razzak, Zongyuan Ge

机构 * Monash University（蒙纳士大学） ； MBZUAI ； XJTLU（西安交通大学） ； Shanghai Jiaotong University（上海交通大学） ； Fudan University（复旦大学） ； University of Minnesota（明尼苏达大学） ； Cornell University（康奈尔大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Clarification note for the CVPR 2025 paper (FarSight). Prepared by a subset of the original authors; remaining co-authors are acknowledged in the text

BYO-Eval: Build Your Own Dataset for Fine-Grained Visual Assessment of Multimodal Language Models

Ludovic Arnould, Salim Khazem, Hugues Ali Mehenni

机构 * R&D Center, Talan（Talan 研发中心） 纠错

专题命中 视觉问答 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

Ontology-based knowledge representation for bone disease diagnosis: a foundation for safe and sustainable medical artificial intelligence systems

Loan Dao, Ngoc Quoc Ly

专题命中 视觉问答 ：visual language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression

Linjie Mu, Zhongzhen Huang, Shengqian Qin, Yakun Zhu, Shaoting Zhang, Xiaofan Zhang

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Question-Answering Dense Video Events

Hangyu Qin, Junbin Xiao, Angela Yao

机构 * National University of Singapore（新加坡国立大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted to SIGIR'25

How Well Can Vison-Language Models Understand Humans' Intention? An Open-ended Theory of Mind Question Evaluation Benchmark

Ximing Wen, Mallika Mainali, Anik Sen

机构 * College of Computing and Informatics, Drexel University（计算与信息学院，德雷塞尔大学） 纠错

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 4 pages, accepted by ToM@AAAI25

VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents

Ryota Tanaka, Taichi Iki, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, Jun Suzuki

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2025; project page: https://vdocrag.github.io

NoTeS-Bank: Benchmarking Neural Transcription and Search for Scientific Notes Understanding

Aniket Pal, Sanket Biswas, Alloy Das, Ayush Lodh, Priyanka Banerjee, Soumitri Chattopadhyay, Dimosthenis Karatzas, Josep Llados, C. V. Jawahar

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification

Laura Fieback, Jakob Spiegelberg, Hanno Gottschalk

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis

Bo Liu, Ke Zou, Liming Zhan, Zexin Lu, Xiaoyu Dong, Yidi Chen, Chengqiang Xie, Jiannong Cao, Xiao-Ming Wu, Huazhu Fu

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments This project is available at https://www.med-vqa.com/GEMeX

Robusto-1 Dataset: Comparing Humans and VLMs on real out-of-distribution Autonomous Driving VQA from Peru

Dunant Cusipuma, David Ortega, Victor Flores-Benites, Arturo Deza

专题命中 视觉问答 ：visual language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments A pre-print. 26 pages. Link to Code + Data: https://huggingface.co/datasets/Artificio/robusto-1

PreMind: Multi-Agent Video Understanding for Advanced Indexing of Presentation-style Videos

Kangda Wei, Zhengyu Zhou, Bingqing Wang, Jun Araki, Lukas Lange, Ruihong Huang, Zhe Feng

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Understanding Figurative Meaning through Explainable Visual Entailment

Arkadiy Saakyan, Shreyas Kulkarni, Tuhin Chakrabarty, Smaranda Muresan

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments NAACL 2025 Main Conference

MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning

Pu Yang, Bin Dong

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Consistency of Compositional Generalization across Multiple Levels

Chuanhao Li, Zhen Li, Chenchen Jing, Xiaomeng Fan, Wenbo Ye, Yuwei Wu, Yunde Jia

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI 2025

CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs

Abhas Kumar, Kapil Pathak, Rajesh Kavuru, Prabhakar Srinivasan

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.AI、cs.LG

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

Te Yang, Jian Jia, Xiangyu Zhu, Weisong Zhao, Bo Wang, Yanhua Cheng, Yan Li, Shengyuan Liu, Quan Chen, Peng Jiang, Kun Gai, Zhen Lei

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering

Guohao Sun, Can Qin, Huazhu Fu, Linwei Wang, Zhiqiang Tao

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments EMNLP 2024 Main conference

Mind the Uncertainty in Human Disagreement: Evaluating Discrepancies between Model Predictions and Human Responses in VQA

Jian Lan, Diego Frassinelli, Barbara Plank

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Parameter Efficient Reinforcement Learning from Human Feedback

Hakim Sidahmed, Samrat Phatale, Alex Hutcheson, Zhuonan Lin, Zhang Chen, Zac Yu, Jarvis Jin, Simral Chaudhary, Roman Komarytsia, Christiane Ahlheim, Yonghao Zhu, Bowen Li, Saravanan Ganesh, Bill Byrne, Jessica Hoffmann, Hassan Mansoor, Wei Li, Abhinav Rastogi, Lucas Dixon

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.AI、cs.LG

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Paper published at AAAI 2024 Spring Symposium Series

Extracting Training Data from Document-Based VQA Models

Francesco Pinto, Nathalie Rauschmayr, Florian Tramèr, Philip Torr, Federico Tombari

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments ICML 2024