视觉大模型 / VLM

CPL: Counterfactual Prompt Learning for Vision and Language Models

Xuehai He, Diji Yang, Weixi Feng, Tsu-Jui Fu, Arjun Akula, Varun Jampani, Pradyumna Narayana, Sugato Basu, William Yang Wang, Xin Eric Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Visual Named Entity Linking: A New Dataset and A Baseline

Wenxiang Sun, Yixing Fan, Jiafeng Guo, Ruqing Zhang, Xueqi Cheng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 11 figures, published to EMNLP 2022(findings)

TVLT: Textless Vision-Language Transformer

Zineng Tang, Jaemin Cho, Yixin Nie, Mohit Bansal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2022 Oral (21 pages; the first three authors contributed equally)

WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models

Yonatan Bitton, Nitzan Bitton Guetta, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2022, Datasets and Benchmarks. Website: https://winogavil.github.io/

LAVIS: A Library for Language-Vision Intelligence

Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, Steven C. H. Hoi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Preprint of LAVIS technical report

Sparse Continuous Distributions and Fenchel-Young Losses

André F. T. Martins, Marcos Treviso, António Farinhas, Pedro M. Q. Aguiar, Mário A. T. Figueiredo, Mathieu Blondel, Vlad Niculae

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Comments JMLR 2022 camera ready version. arXiv admin note: text overlap with arXiv:2006.07214

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Patrick Schramowski, Christopher Tauchmann, Kristian Kersting

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2110.04222

Less Is More: Linear Layers on CLIP Features as Powerful VizWiz Model

Fabian Deuser, Konrad Habel, Philipp J. Rösch, Norbert Oswald

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments VizWiz Grand Challenge: Describing Images and Videos Taken by Blind People (CVPR Workshop 2022)

V-Doc : Visual questions answers with Documents

Yihao Ding, Zhe Huang, Runlin Wang, Yanhang Zhang, Xianru Chen, Yuzhong Ma, Hyunsuk Chung, Soyeon Caren Han

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2022

Learning to Answer Visual Questions from Web Videos

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted at the TPAMI Special Issue on the Best Papers of ICCV 2021. Journal extension of the conference paper arXiv:2012.00451. 16 pages, 13 figures

Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering

Min Peng, Chongyang Wang, Yuan Gao, Yu Shi, Xiang-Dong Zhou

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IJCAI 2022. arXiv admin note: text overlap with arXiv:2109.04735

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACL 2022

Can you even tell left from right? Presenting a new challenge for VQA

Sai Raam Venkatraman, Rishi Rao, S. Balasubramanian, Chandra Sekhar Vorugunti, R. Raghunatha Sarma

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Towards Automated Error Analysis: Learning to Characterize Errors

Tong Gao, Shivang Singh, Raymond J. Mooney

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 11 figures

Can Open Domain Question Answering Systems Answer Visual Knowledge Questions?

Jiawen Zhang, Abhijit Mishra, Avinesh P. V. S, Siddharth Patwardhan, Sachin Agarwal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 9 pages (including references), 5 figures

How Modular Should Neural Module Networks Be for Systematic Generalization?

Vanessa D'Amario, Tomotake Sasaki, Xavier Boix

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Journal ref 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Sydney, Australia

COIN: Counterfactual Image Generation for VQA Interpretation

Zeyd Boukhers, Timo Hartmann, Jan Jürjens

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation

Tianyi Liu, Zuxuan Wu, Wenhan Xiong, Jingjing Chen, Yu-Gang Jiang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Playing Lottery Tickets with Vision and Language

Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted to AAAI 2022

MMIU: Dataset for Visual Intent Understanding in Multimodal Assistants

Alkesh Patel, Joel Ruben Antony Moniz, Roman Nguyen, Nick Tzou, Hadas Kotek, Vincent Renkens

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Extended abstract accepted for WeCNLP 2021

Generating and Evaluating Explanations of Attended and Error-Inducing Input Regions for VQA Models

Arijit Ray, Michael Cogswell, Xiao Lin, Kamran Alipour, Ajay Divakaran, Yi Yao, Giedrius Burachas

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Applied AI Letters, Wiley, 25 October 2021

Improving Users' Mental Model with Attention-directed Counterfactual Edits

Kamran Alipour, Arijit Ray, Xiao Lin, Michael Cogswell, Jurgen P. Schulze, Yi Yao, Giedrius T. Burachas

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication in Applied AI Letters

Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions

Daniel Rosenberg, Itai Gat, Amir Feder, Roi Reichart

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments ACL 2021. Our code and data are available at https://danrosenberg.github.io/rad-measure/

Just Ask: Learning to Answer Questions from Millions of Narrated Videos

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted at ICCV 2021 (Oral); 20 pages; 14 figures

BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosis

Masoud Monajatipoor, Mozhdeh Rouhsedaghat, Liunian Harold Li, Aichi Chien, C. -C. Jay Kuo, Fabien Scalzo, Kai-Wei Chang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 8 figures, Accepted in ICCV workshop

Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering

Ahjeong Seo, Gi-Cheon Kang, Joonhan Park, Byoung-Tak Zhang

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments ACL 2021

Beyond VQA: Generating Multi-word Answer and Rationale to Visual Questions

Radhika Dua, Sai Srinivas Kancheti, Vineeth N Balasubramanian

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments MULA Workshop, CVPR 2021

PAM: Understanding Product Images in Cross Product Category Attribute Extraction

Rongmei Lin, Xiang He, Jie Feng, Nasser Zalmout, Yan Liang, Li Xiong, Xin Luna Dong

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments KDD 2021

Neuro-Symbolic VQA: A review from the perspective of AGI desiderata

Ian Berlot-Attwell

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Multimodal Continuous Visual Attention Mechanisms

António Farinhas, André F. T. Martins, Pedro M. Q. Aguiar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG