视觉大模型 / VLM

A Vision Centric Remote Sensing Benchmark

Abduljaleel Adejumo, Faegheh Yeganli, Clifford Broni-bediako, Aoran Xiao, Naoto Yokoya, Mennatullah Siam

机构 * AMMI/AIMS Senegal（AMMI/AIMS塞内加尔） ； University of British Columbia（不列颠哥伦比亚大学） ； RIKEN AIP（理化学研究所AIP） ； the University of Tokyo（东京大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Eval-FoMo2 Workshop in CVPR 2025

Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding

Tao Zhang, Xiangtai Li, Zilong Huang, Yanwei Li, Weixian Lei, Xueqing Deng, Shihao Chen, Shunping Ji, Jiashi Feng

专题命中 视觉问答 ：vision-language model(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

AstroLLaVA: towards the unification of astronomical data and natural language

Sharaf Zaman, Michael J. Smith, Pranav Khetarpal, Rishabh Chakrabarty, Michele Ginolfi, Marc Huertas-Company, Maja Jabłońska, Sandor Kruk, Matthieu Le Lain, Sergio José Rodríguez Méndez, Dimitrios Tanoglidis

专题命中 视觉问答 ：vision language model(abstract)；LLaVA(abstract)；visual question answering(abstract)；分类 cs.LG

Comments 8 pages, 3 figures, accepted to SCI-FM@ICLR 2025. Code at https://w3id.org/UniverseTBD/AstroLLaVA

FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs

Xiaoqin Wang, Xusen Ma, Xianxu Hou, Meidan Ding, Yudong Li, Junliang Chen, Wenting Chen, Xiaoyang Peng, Linlin Shen

专题命中 视觉问答 ：LLaVA(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by CVPR2025

Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework

Zhuo Zhi, Chen Feng, Adam Daneshmend, Mine Orlu, Andreas Demosthenous, Lu Yin, Da Li, Ziquan Liu, Miguel R. D. Rodrigues

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

A Benchmark for Crime Surveillance Video Analysis with Large Models

Haoran Chen, Dong Yi, Moyan Cao, Chensen Huang, Guibo Zhu, Jinqiao Wang

专题命中 视觉问答 ：LLaVA(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity

Hang Hua, Qing Liu, Lingzhi Zhang, Jing Shi, Zhifei Zhang, Yilin Wang, Jianming Zhang, Jiebo Luo

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Preprint

VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding

Xiang Li, Jian Ding, Mohamed Elhoseiny

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted for publication at NeruIPS 2024 Dataset and Benchmark Track

ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue

Zhangpu Li, Changhong Zou, Suxue Ma, Zhicheng Yang, Chen Du, Youbao Tang, Zhenjie Cao, Ning Zhang, Jui-Hsin Lai, Ruei-Sung Lin, Yuan Ni, Xingzhi Sun, Jing Xiao, Jieke Hou, Kai Zhang, Mei Han

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation

Anna C. Doris, Daniele Grandi, Ryan Tomich, Md Ferdous Alam, Mohammadmehdi Ataei, Hyunmin Cheong, Faez Ahmed

专题命中 视觉问答 ：LLaVA(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation

Yuxuan Wang, Yijun Liu, Fei Yu, Chen Huang, Kexin Li, Zhiguo Wan, Wanxiang Che

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers

Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang, Chun Yuan, Jiaqi Wang

专题命中 视觉问答 ：vision-language model(abstract)；visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments ICML 2024. Code: https://github.com/sdc17/CrossGET

HOI-Ref: Hand-Object Interaction Referral in Egocentric Vision

Siddhant Bansal, Michael Wray, Dima Damen

专题命中 视觉问答 ：vision language model(abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Project Page: https://sid2697.github.io/hoi-ref/

Visual Hallucination: Definition, Quantification, and Prescriptive Remediations

Anku Rani, Vipula Rawte, Harshad Sharma, Neeraj Anand, Krishnav Rajbangshi, Amit Sheth, Amitava Das

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.AI

InfMLLM: A Unified Framework for Visual-Language Tasks

Qiang Zhou, Zhibin Wang, Wei Chu, Yinghui Xu, Hao Li, Yuan Qi

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments 8

What Large Language Models Bring to Text-rich VQA?

Xuejing Liu, Wei Tang, Xinzhe Ni, Jinghui Lu, Rui Zhao, Zechao Li, Fei Tan

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Lightweight In-Context Tuning for Multimodal Unified Models

Yixin Chen, Shuai Zhang, Boran Han, Jiaya Jia

专题命中 视觉问答 ：LLaVA(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Preprint

What Makes for Good Visual Tokenizers for Large Language Models?

Guangzhi Wang, Yixiao Ge, Xiaohan Ding, Mohan Kankanhalli, Ying Shan

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments 15 pages, 3 figures. Project released at: https://github.com/TencentARC/GVT

Language Is Not All You Need: Aligning Perception with Language Models

Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

Yibing Liu, Yangyang Guo, Jianhua Yin, Xuemeng Song, Weifeng Liu, Liqiang Nie

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments ACM TOMM 2021

What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation

时间基准分数衡量的是什么？分解视频视觉语言模型评估中的通道使用情况

Farrukh Rahman

机构 * Microsoft（微软） ； Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 视觉问答 ：VLM(title)；分类 cs.CV、cs.LG

AI总结 研究视频视觉语言模型评估中时间基准分数衡量问题，提出无标签筛选方法“反转下降”区分模型通道使用情况，如Molmo2和Qwen3-VL，指出综合分数不能反映潜在失败模式，此区分在多基准和任务中成立。

Comments 9 pages, 11 pages supplemental

Location-Aware Pretraining for Medical Difference Visual Question Answering

面向位置的预训练方法用于医学差异视觉问答

Denis Musinguzi, Caren Han, Prasenjit Mitra

机构 * Department of Electrical and Computer Engineering, Carnegie Mellon University, Kigali, Rwanda（电气与计算机工程系，卡内基梅隆大学，刚果（金）基利齐） ； University of Melbourne, Melbourne, Australia（墨尔本大学，墨尔本，澳大利亚） 纠错

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI

AI总结 本文提出一种面向位置的预训练框架，结合AREF、GCAP和CAREF任务，提升医学差异VQA中细粒度空间视觉表征能力，实现胸部X光图像中临床相关变化的准确识别与推理。

Comments 11 pages

Surgical-MambaLLM: Mamba2-enhanced Multimodal Large Language Model for VQLA in Robotic Surgery

Pengfei Hao, Hongqiu Wang, Shuaibo Li, Zhaohu Xing, Guang Yang, Kaishun Wu, Lei Zhu

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； Imperial College London（帝国理工学院伦敦分校） ； The Hong Kong University of Science and Technology（香港科学与技术大学） 纠错

专题命中 视觉问答 ：multimodal large language model(title)；分类 cs.CV、cs.AI

Comments Early accepted by MICCAI2025

Quantifying Memorization and Parametric Response Rates in Retrieval-Augmented Vision-Language Models

Peter Carragher, Abhinand Jha, R Raghav, Kathleen M. Carley

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.AI、cs.LG

Hallucination Benchmark in Medical Visual Question Answering

Jinge Wu, Yunsoo Kim, Honghan Wu

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2024 Tiny Papers(Notable)

Analyzing Zero-Shot Abilities of Vision-Language Models on Video Understanding Tasks

Avinash Madasu, Anahita Bhiwandiwalla, Vasudev Lal

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV、cs.AI

LXMERT Model Compression for Visual Question Answering

Maryam Hashemi, Ghazaleh Mahmoudi, Sara Kodeiri, Hadi Sheikhi, Sauleh Eetemadi

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.LG

Comments To appear in The Fourth Annual West Coast NLP (WeCNLP) Summit

Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models

Guangzhi Sun, Wenyi Yu, Changli Tang, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Chao Zhang

专题命中 视觉问答 ：multimodal large language model(title)；分类 cs.CV、cs.AI

Visual Question Answering (VQA) on Images with Superimposed Text

Venkat Kodali, Daniel Berleant

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI

Comments To appear in: Proc. of the Future Technologies Conference (FTC) 2023, Nov. 2-3, San Francisco, pub. by Springer Nature

Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering

Long Chen, Yuhang Zheng, Yulei Niu, Hanwang Zhang, Jun Xiao

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI

Comments IEEE Transactions on Pattern Analysis and Machine Intelligence, TPAMI 2023. (Extension of CVPR'20 work). arXiv admin note: text overlap with arXiv:2003.06576