视觉大模型 / VLM

EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought

Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, Ping Luo

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

DLIP: Distilling Language-Image Pre-training

Huafeng Kuang, Jie Wu, Xiawu Zheng, Ming Li, Xuefeng Xiao, Rui Wang, Min Zheng, Rongrong Ji

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

I2I: Initializing Adapters with Improvised Knowledge

Tejas Srinivasan, Furong Jia, Mohammad Rostami, Jesse Thomason

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at 2nd Conference on Lifelong Learning Agents (CoLLAs), 2023

GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions

Woojeong Jin, Subhabrata Mukherjee, Yu Cheng, Yelong Shen, Weizhu Chen, Ahmed Hassan Awadallah, Damien Jose, Xiang Ren

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)

Comments Preprint

A Study of Autoregressive Decoders for Multi-Tasking in Computer Vision

Lucas Beyer, Bo Wan, Gagan Madan, Filip Pavetic, Andreas Steiner, Alexander Kolesnikov, André Susano Pinto, Emanuele Bugliarello, Xiao Wang, Qihang Yu, Liang-Chieh Chen, Xiaohua Zhai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Optimizing Explanations by Network Canonization and Hyperparameter Search

Frederik Pahde, Galip Ümit Yolcu, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Few-shot Multimodal Multitask Multilingual Learning

Aman Chadha, Vinija Jain

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

Oscar Mañas, Pau Rodriguez, Saba Ahmadi, Aida Nematzadeh, Yash Goyal, Aishwarya Agrawal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at EACL 2023 (main track); 26 pages, 21 figures, 6 tables; Pau Rodriguez and Saba Ahmadi had equal contributions

SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models

Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, Animesh Garg

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICLR 2023. Project page: https://slotformer.github.io/

CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks

Zhecan Wang, Noel Codella, Yen-Chun Chen, Luowei Zhou, Jianwei Yang, Xiyang Dai, Bin Xiao, Haoxuan You, Shih-Fu Chang, Lu Yuan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This paper is greatly modified and updated to be re-submitted to another conference. The new paper is under the name "Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks", https://doi.org/10.48550/arXiv.2204.10496

AI2-THOR: An Interactive 3D Environment for Visual AI

Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, Aniruddha Kembhavi, Abhinav Gupta, Ali Farhadi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers

Estelle Aflalo, Meng Du, Shao-Yen Tseng, Yongfei Liu, Chenfei Wu, Nan Duan, Vasudev Lal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Best Demo Award at CVPR 2022

American == White in Multimodal Language-and-Image AI

Robert Wolfe, Aylin Caliskan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to AI Ethics and Society 2022

VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks

Hung Le, Nancy F. Chen, Steven C. H. Hoi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at NAACL 2022 (Oral)

Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks

Zhecan Wang, Noel Codella, Yen-Chun Chen, Luowei Zhou, Xiyang Dai, Bin Xiao, Jianwei Yang, Haoxuan You, Kai-Wei Chang, Shih-fu Chang, Lu Yuan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2201.05729

Towards General Purpose Vision Systems

Tanmay Gupta, Amita Kamath, Aniruddha Kembhavi, Derek Hoiem

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2022 Oral; Project page: https://prior.allenai.org/projects/gpv

Co-VQA : Answering by Interactive Sub Question Sequence

Ruonan Wang, Yuxi Qian, Fangxiang Feng, Xiaojie Wang, Huixing Jiang

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)

Comments Accepted by Findings of ACL 2022

WebQA: Multihop and Multimodal QA

Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, Yonatan Bisk

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR Camera ready

X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics

Yehao Li, Yingwei Pan, Jingwen Chen, Ting Yao, Tao Mei

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by 2021 ACMMM Open Source Software Competition. Source code: https://github.com/YehLi/xmodaler

How Much Can CLIP Benefit Vision-and-Language Tasks?

Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 14 pages

Unifying Vision-and-Language Tasks via Text Generation

Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2021 (15 pages, 4 figures, 14 tables)

A survey on VQA_Datasets and Approaches

Yeyun Zou, Qiyu Xie

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 10 pages

Generating Natural Questions from Images for Multimodal Assistants

Alkesh Patel, Akanksha Bindal, Hadas Kotek, Christopher Klein, Jason Williams

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 4 pages, 1 reference page, 5 figures, 4 tables

Large Scale Multimodal Classification Using an Ensemble of Transformer Models and Co-Attention

Varnith Chordia, Vijay Kumar BG

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Answering Questions about Data Visualizations using Efficient Bimodal Fusion

Kushal Kafle, Robik Shrestha, Brian Price, Scott Cohen, Christopher Kanan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Presented at WACV, 2020

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

Chao Zhang, Zichao Yang, Xiaodong He, Li Deng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Factor Graph Attention

Idan Schwartz, Seunghak Yu, Tamir Hazan, Alexander Schwing

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2019; revised version includes bottom-up features

What Can Neural Networks Reason About?

Keyulu Xu, Jingling Li, Mozhi Zhang, Simon S. Du, Ken-ichi Kawarabayashi, Stefanie Jegelka

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Deep Exemplar Networks for VQA and VQG

Badri N. Patro, Vinay P. Namboodiri

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This work is an extension of CVPR-2018 accepted paper arXiv:1804.00298 and EMNLP-2018 accepted paper arXiv:1808.03986

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This version was published in International Journal of Computer Vision (IJCV) in 2019; A previous version of the paper was published at International Conference on Computer Vision (ICCV'17)