视觉大模型 / VLM - arXivDaily 专题

2305.12397 2023-12-11 cs.CV 70%

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

Yuanyuan Jiang, Jianqin Yin

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted to EMNLP 2023 Findings

2311.17043 2023-11-29 cs.CV cs.CL 70%

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

Yanwei Li, Chengyao Wang, Jiaya Jia

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Code is available at https://github.com/dvlab-research/LLaMA-VID

2310.09478 2023-11-08 cs.CV 70%

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, Mohamed Elhoseiny

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2310.19145 2023-10-31 cs.CL cs.CV 70%

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Tuhin Chakrabarty, Kanishk Singh, Arkadiy Saakyan, Smaranda Muresan

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Findings of EMNLP 2023 (Long paper)

2310.19061 2023-10-31 cs.CV 70%

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

Zhiling Yan, Kai Zhang, Rong Zhou, Lifang He, Xiang Li, Lichao Sun

专题命中视觉问答：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV

2307.00716 2023-10-31 cs.CV 70%

JourneyDB: A Benchmark for Generative Image Understanding

Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Limin Wang, Hongsheng Li

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

2310.17050 2023-10-27 cs.CV 70%

Exploring Question Decomposition for Zero-Shot VQA

Zaid Khan, Vijay Kumar BG, Samuel Schulter, Manmohan Chandraker, Yun Fu

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments NeurIPS 2023 Camera Ready

2308.09033 2023-09-20 cs.CV 70%

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Fawaz Sammani, Nikos Deligiannis

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCVW 2023

2308.10354 2023-08-22 cs.AI cs.CL 70%

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Zeinab Sadat Taghavi, Soroush Gooran, Seyed Arshan Dalili, Hamidreza Amirzadeh, Mohammad Jalal Nematbakhsh, Hossein Sameti

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments 18 pages,

2307.09416 2023-07-20 cs.CV cs.CL 70%

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted as oral at ACM MultiMedia 2023 (Brave New Ideas track)

2205.12005 2023-07-06 cs.CL cs.CV 70%

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou, Luo Si

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Journal ref EMNLP2022

2305.16328 2023-05-29 cs.CL cs.LG 70%

Semantic Composition in Visually Grounded Language Models

Rohan Pandey

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.LG

Comments Carnegie Mellon University Senior Thesis. arXiv admin note: substantial text overlap with arXiv:2212.10549

2303.02635 2023-03-07 cs.CV cs.MM 70%

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Kang Chen, Xiangqian Wu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

2208.09374 2022-08-22 cs.CV 70%

VLMAE: Vision-Language Masked Autoencoder

Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, Bo Ren

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

2208.03030 2022-08-08 cs.CL cs.CV 70%

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Bingning Wang, Feiyang Lv, Ting Yao, Yiming Yuan, Jin Ma, Yu Luo, Haijin Liang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments CIKM2022 camera ready version

2201.10963 2022-05-31 cs.CV cs.MM 70%

Learning to Compose Diversified Prompts for Image Emotion Classification

Sinuo Deng, Lifang Wu, Ge Shi, Lehao Xing, Meng Jian, Ye Xiang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 10 pages, 5 figures

2106.13488 2021-11-10 cs.CV 70%

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2021

2108.08965 2021-08-23 cs.CV cs.CL 70%

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Xiaopeng Lu, Zhen Fan, Yansen Wang, Jean Oh, Carolyn P. Rose

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 9 pages

1911.09655 2019-11-22 cs.CL cs.LG cs.SD eess.AS 70%

Temporal Reasoning via Audio Question Answering

Haytham M. Fayek, Justin Johnson

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.LG

1905.06139 2019-11-05 cs.CL cs.CV 70%

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Fenglin Liu, Yuanxin Liu, Xuancheng Ren, Xiaodong He, Xu Sun

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2019

1902.03751 2019-10-29 cs.CV 70%

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, Devi Parikh

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Published at ICCV'2019

Journal ref The IEEE International Conference on Computer Vision (ICCV) 2019

1902.09368 2019-08-30 cs.CV 70%

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Gi-Cheon Kang, Jaeseo Lim, Byoung-Tak Zhang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments EMNLP 2019

1908.06327 2019-08-20 cs.CV cs.CL 70%

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments ICCV 2019 accepted paper

1908.04107 2019-08-20 cs.CV 70%

Multimodal Unified Attention Networks for Vision-and-Language Interactions

Zhou Yu, Yuhao Cui, Jun Yu, Dacheng Tao, Qi Tian

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 11 pages, 7 figures

1907.06794 2019-08-20 cs.CV 70%

2nd Place Solution to the GQA Challenge 2019

Shijie Geng, Ji Zhang, Hang Zhang, Ahmed Elgammal, Dimitris N. Metaxas

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1901.06706 2019-01-23 cs.CV 70%

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Ning Xie, Farley Lai, Derek Doran, Asim Kadav

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1812.00500 2018-12-04 cs.CV 70%

Multi-task Learning of Hierarchical Vision-Language Representation

Duy-Kien Nguyen, Takayuki Okatani

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

1709.07992 2018-08-08 cs.CV 70%

Visual Reference Resolution using Attention Memory for Visual Dialog

Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, Leonid Sigal

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2608.06571 2026-08-10 cs.CR cs.CL 新提交 67%

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier

答案保留型攻击下的模型置信度：信息性-可操纵性前沿

Reza Khanmohammadi, Ivan Brugere, Simerjot Kaur, Charese H. Smiley, Kundan Thind, Mohammad M. Ghassemi

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)

AI总结该研究针对答案保留型攻击，发现视觉-语言系统的置信度信号不具备固有鲁棒性，四类防御均无效，协调攻击可大幅降低置信度门控下的接受准确率。

1. 视觉问答 3129 篇

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

JourneyDB: A Benchmark for Generative Image Understanding

Exploring Question Decomposition for Zero-Shot VQA

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Semantic Composition in Visually Grounded Language Models

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Champion Solution for the WSDM2023 Toloka VQA Challenge

VLMAE: Vision-Language Masked Autoencoder

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Learning to Compose Diversified Prompts for Image Emotion Classification

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Temporal Reasoning via Audio Question Answering

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Multimodal Unified Attention Networks for Vision-and-Language Interactions

2nd Place Solution to the GQA Challenge 2019

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Multi-task Learning of Hierarchical Vision-Language Representation

Visual Reference Resolution using Attention Memory for Visual Dialog

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier