视觉大模型 / VLM

WeatherQA: Can Multimodal Language Models Reason about Severe Weather?

Chengqian Ma, Zhanxiang Hua, Alexandra Anderson-Frey, Vikram Iyer, Xin Liu, Lianhui Qin

专题命中 视觉问答 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 26 pages, 9 figures

AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding

Alessandro Suglia, Claudio Greco, Katie Baker, Jose L. Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Code available https://github.com/alanaai/EVUD

Wings: Learning Multimodal LLMs without Text-only Forgetting

Yi-Kai Zhang, Shiyin Lu, Yang Li, Yanqing Ma, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Bridging Vision and Language Spaces with Assignment Prediction

Jungin Park, Jiyoung Lee, Kwanghoon Sohn

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments ICLR 2024 Camera-ready

Can I Trust Your Answer? Visually Grounded Video Question Answering

Junbin Xiao, Angela Yao, Yicong Li, Tat Seng Chua

专题命中 视觉问答 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR'24. (Compared with preprint version, we mainly improve the presentation, discuss more related works, and extend experiments in Appendix.)

A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis

Yingshu Li, Yunyi Liu, Zhanyu Wang, Xinyu Liang, Lei Wang, Lingqiao Liu, Leyang Cui, Zhaopeng Tu, Longyue Wang, Luping Zhou

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

MIVC: Multiple Instance Visual Component for Visual-Language Models

Wenyi Wu, Qi Li, Wenliang Zhong, Junzhou Huang

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted at WACV 2024

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs

Shengzhi Li, Nima Tajbakhsh

专题命中 视觉问答 ：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Med-Flamingo: a Multimodal Medical Few-shot Learner

Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, Jure Leskovec

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Preprint

Is Multimodal Vision Supervision Beneficial to Language?

Avinash Madasu, Vasudev Lal

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory

Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, Alireza Fathi

专题命中 视觉问答 ：visual language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Published on CVPR 2023

Towards Models that Can See and Read

Roy Ganz, Oren Nuriel, Aviad Aberdam, Yair Kittenplon, Shai Mazor, Ron Litman

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

PaLM-E: An Embodied Multimodal Language Model

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.AI、cs.LG

Compound Tokens: Channel Fusion for Vision-Language Representation Learning

Maxwell Mbabilla Aladago, AJ Piergiovanni

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, Jianlong Fu

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

12-in-1: Multi-Task Vision and Language Representation Learning

Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, Stefan Lee

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Jiasen Lu and Vedanuj Goswami contributed equally to this work

Modulated Self-attention Convolutional Network for VQA

Jean-Benoit Delbrouck, Antoine Maiorca, Nathan Hubens, Stéphane Dupont

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted at NeurIPS 2019 workshop: ViGIL

Visual Coreference Resolution in Visual Dialog using Neural Module Networks

Satwik Kottur, José M. F. Moura, Devi Parikh, Dhruv Batra, Marcus Rohrbach

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments ECCV 2018 + results on VisDial v1.0 dataset

Visual7W: Grounded Question Answering in Images

Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments CVPR 2016

Explicit Evidence Grounding via Structured Inline Citation Generation

通过结构化内联引文生成实现显式证据基础

Anar Yeginbergen, Amelie Wührl, Anna Rogers, Rodrigo Agerri

机构 * University of the Basque Country (UPV/EHU)（巴斯克大学） ； IT University of Copenhagen（哥本哈根IT大学） 纠错

专题命中 视觉问答 ：grounding(title)

AI总结 提出FullCite框架，通过提示生成、约束解码和后处理跨度对齐三种策略生成结构化内联引文，在三个QA基准上评估引文质量和忠实性，发现LLMs虽能识别相关文档但难以精确定位支持性证据跨度。

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering

耶鲁-DM实验室参加ArchEHR-QA 2026：用于电子病历问答的确定性接地和多轮证据对齐

Elyas Irankhah, Samah Fodeh

机构 * Yale University（耶鲁大学） 纠错

专题命中 视觉问答 ：grounding(title)

AI总结 本文提出确定性接地和多轮证据对齐方法，用于电子病历问答任务，通过模型多样性与投票策略提升性能，实验结果显示对齐准确率受限于推理能力。

Comments 9 pages, 2 figures. System description for ArchEHR-QA 2026 shared task

Automated Bug Frame Retrieval from Gameplay Videos Using Vision-Language Models

Wentao Lu, Alexander Senchenko, Abram Hindle, Cor-Paul Bezemer

专题命中 视觉问答 ：vision-language model(title)

The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systems

Judith Sieker, Simeon Junker, Ronja Utescher, Nazia Attari, Heiko Wersing, Hendrik Buschmeier, Sina Zarrieß

机构 * Computational Linguistics, Department of Linguistics, Bielefeld University（语言学计算系，语言学系，比勒菲尔德大学） ； Honda Research Institute Europe（本田欧洲研究院） ； Digital Linguistics Lab, Department of Linguistics, Bielefeld University（数字语言学实验室，语言学系，比勒菲尔德大学） 纠错

专题命中 视觉问答 ：visual question answering(title)

Comments 17 pages (including Appendix). Accepted at EMNLP 2024 main

Journal ref Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 19459-19475

SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models

Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang, Jiaheng Liu, Yuying Mai, Yutao Zeng, Zhoufutu Wen, Ke Jin, Baorui Wang, Weixiao Zhou, Yunhong Lu, Tongliang Li, Wenhao Huang, Zhoujun Li

专题命中 视觉问答 ：multimodal large language model(title)

PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese

Nghia Hieu Nguyen, Kiet Van Nguyen

专题命中 视觉问答 ：visual question answering(title)

Structured Knowledge Grounding for Question Answering

Yujie Lu, Siqi Ouyang, Kairui Zhou

专题命中 视觉问答 ：grounding(title)

MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models

Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin F. Yang, Kai-Wei Chang

专题命中 视觉问答 ：vision-language model(title)

CHIC: Corporate Document for Visual question Answering

Ibrahim Souleiman Mahamoud, Mickael Coustaty, Aurelie Joseph, Vincent Poulain d Andecy, Jean-Marc Ogier

专题命中 视觉问答 ：visual question answering(title)

From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathology

从多分辨率细胞到千兆像素全切片图像：用于计算病理学的基础模型

Basit Alawode, Moshira Ali Abdalla, Dwarikanath Mahapatra, Muzammal Naseer, Sajid Javed

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV

AI总结 针对现有计算病理学模型泛化性受限的问题，提出多分辨率金字塔Transformer（MRPT），经多分辨率自监督预训练后，在34个数据集的多项任务中性能优于同类模型与多模态大语言模型。