视觉大模型 / VLM - arXivDaily 专题

2310.09478 2023-11-08 cs.CV 70%

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, Mohamed Elhoseiny

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2310.19145 2023-10-31 cs.CL cs.CV 70%

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Tuhin Chakrabarty, Kanishk Singh, Arkadiy Saakyan, Smaranda Muresan

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Findings of EMNLP 2023 (Long paper)

2310.19061 2023-10-31 cs.CV 70%

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

Zhiling Yan, Kai Zhang, Rong Zhou, Lifang He, Xiang Li, Lichao Sun

专题命中视觉问答：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV

2307.00716 2023-10-31 cs.CV 70%

JourneyDB: A Benchmark for Generative Image Understanding

Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Limin Wang, Hongsheng Li

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

2310.17050 2023-10-27 cs.CV 70%

Exploring Question Decomposition for Zero-Shot VQA

Zaid Khan, Vijay Kumar BG, Samuel Schulter, Manmohan Chandraker, Yun Fu

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments NeurIPS 2023 Camera Ready

2308.09033 2023-09-20 cs.CV 70%

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Fawaz Sammani, Nikos Deligiannis

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCVW 2023

2308.10354 2023-08-22 cs.AI cs.CL 70%

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Zeinab Sadat Taghavi, Soroush Gooran, Seyed Arshan Dalili, Hamidreza Amirzadeh, Mohammad Jalal Nematbakhsh, Hossein Sameti

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments 18 pages,

2307.09416 2023-07-20 cs.CV cs.CL 70%

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted as oral at ACM MultiMedia 2023 (Brave New Ideas track)

2205.12005 2023-07-06 cs.CL cs.CV 70%

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou, Luo Si

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Journal ref EMNLP2022

2305.16328 2023-05-29 cs.CL cs.LG 70%

Semantic Composition in Visually Grounded Language Models

Rohan Pandey

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.LG

Comments Carnegie Mellon University Senior Thesis. arXiv admin note: substantial text overlap with arXiv:2212.10549

2303.02635 2023-03-07 cs.CV cs.MM 70%

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Kang Chen, Xiangqian Wu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

2208.09374 2022-08-22 cs.CV 70%

VLMAE: Vision-Language Masked Autoencoder

Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, Bo Ren

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

2208.03030 2022-08-08 cs.CL cs.CV 70%

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Bingning Wang, Feiyang Lv, Ting Yao, Yiming Yuan, Jin Ma, Yu Luo, Haijin Liang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments CIKM2022 camera ready version

2201.10963 2022-05-31 cs.CV cs.MM 70%

Learning to Compose Diversified Prompts for Image Emotion Classification

Sinuo Deng, Lifang Wu, Ge Shi, Lehao Xing, Meng Jian, Ye Xiang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 10 pages, 5 figures

2106.13488 2021-11-10 cs.CV 70%

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2021

2108.08965 2021-08-23 cs.CV cs.CL 70%

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Xiaopeng Lu, Zhen Fan, Yansen Wang, Jean Oh, Carolyn P. Rose

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 9 pages

1911.09655 2019-11-22 cs.CL cs.LG cs.SD eess.AS 70%

Temporal Reasoning via Audio Question Answering

Haytham M. Fayek, Justin Johnson

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.LG

1905.06139 2019-11-05 cs.CL cs.CV 70%

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Fenglin Liu, Yuanxin Liu, Xuancheng Ren, Xiaodong He, Xu Sun

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2019

1902.03751 2019-10-29 cs.CV 70%

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, Devi Parikh

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Published at ICCV'2019

Journal ref The IEEE International Conference on Computer Vision (ICCV) 2019

1902.09368 2019-08-30 cs.CV 70%

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Gi-Cheon Kang, Jaeseo Lim, Byoung-Tak Zhang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments EMNLP 2019

1908.06327 2019-08-20 cs.CV cs.CL 70%

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments ICCV 2019 accepted paper

1908.04107 2019-08-20 cs.CV 70%

Multimodal Unified Attention Networks for Vision-and-Language Interactions

Zhou Yu, Yuhao Cui, Jun Yu, Dacheng Tao, Qi Tian

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 11 pages, 7 figures

1907.06794 2019-08-20 cs.CV 70%

2nd Place Solution to the GQA Challenge 2019

Shijie Geng, Ji Zhang, Hang Zhang, Ahmed Elgammal, Dimitris N. Metaxas

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1901.06706 2019-01-23 cs.CV 70%

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Ning Xie, Farley Lai, Derek Doran, Asim Kadav

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1812.00500 2018-12-04 cs.CV 70%

Multi-task Learning of Hierarchical Vision-Language Representation

Duy-Kien Nguyen, Takayuki Okatani

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

1709.07992 2018-08-08 cs.CV 70%

Visual Reference Resolution using Attention Memory for Visual Dialog

Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, Leonid Sigal

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2608.06571 2026-08-10 cs.CR cs.CL 新提交 67%

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier

答案保留型攻击下的模型置信度：信息性-可操纵性前沿

Reza Khanmohammadi, Ivan Brugere, Simerjot Kaur, Charese H. Smiley, Kundan Thind, Mohammad M. Ghassemi

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)

AI总结该研究针对答案保留型攻击，发现视觉-语言系统的置信度信号不具备固有鲁棒性，四类防御均无效，协调攻击可大幅降低置信度门控下的接受准确率。

2604.00086 2026-08-04 cs.CV cs.AI cs.CL cs.LG 版本更新 67%

Hierarchical Pre-Training of Vision Encoders with Large Language Model

基于大语言模型的视觉编码器分层预训练

Eugene Lee, Ting-Yu Chang, Jui-Huang Tsai, Jiajie Diao, Chen-Yi Lee

机构 * University of Cincinnati（辛辛那提大学）； National Yang Ming Chiao Tung University（国立阳明交通大学）

专题命中视觉问答：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出HIVE框架，通过引入视觉编码器与大语言模型间的分层交叉注意力机制，提升视觉语言对齐，改进特征融合与表征学习，实验表明其在图像分类和多模态任务中表现优异。

Comments 17 pages, 14 figures, accepted to Computer Vision and Pattern Recognition Conference (CVPR) Workshops 2026. 5th MMFM Workshop: What is Next in Multimodal Foundation Models?

Journal ref In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7415-7424) 2026

2605.09635 2026-07-24 cs.CL 版本更新 67%

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

K12-KGraph：一种对齐课程的图谱用于基准测试和训练教育大语言模型

Hao Liang, Qihan Lin, Zhaoyang Han, Xiaochen Ma, Zhen Hao Wong, Meiyi Qiang, Linzhuang Sun, Wentao Zhang

机构 * Peking University（北京大学）； Institute for Advanced Algorithms Research（先进算法研究所）； OriginHub Technology（OriginHub技术）； Zhongguancun Academy（中关村学院）

专题命中视觉问答：vision-language model(abstract)；grounding(abstract)

AI总结本文提出K12-KGraph，基于中小学教材构建的课程对齐知识图谱，用于构建多选基准测试K12-Bench和训练数据集K12-Train，验证课程结构监督在教育模型训练中的高效性。

1. 视觉问答 3126 篇

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

JourneyDB: A Benchmark for Generative Image Understanding

Exploring Question Decomposition for Zero-Shot VQA

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Semantic Composition in Visually Grounded Language Models

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Champion Solution for the WSDM2023 Toloka VQA Challenge

VLMAE: Vision-Language Masked Autoencoder

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Learning to Compose Diversified Prompts for Image Emotion Classification

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Temporal Reasoning via Audio Question Answering

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Multimodal Unified Attention Networks for Vision-and-Language Interactions

2nd Place Solution to the GQA Challenge 2019

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Multi-task Learning of Hierarchical Vision-Language Representation

Visual Reference Resolution using Attention Memory for Visual Dialog

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier

Hierarchical Pre-Training of Vision Encoders with Large Language Model

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs