视觉大模型 / VLM

Single Sequence Prediction over Reasoning Graphs for Multi-hop QA

Gowtham Ramesh, Makesh Sreedhar, Junjie Hu

专题命中 视觉问答 ：grounding(abstract)；分类 cs.LG

Diversifying Joint Vision-Language Tokenization Learning

Vardaan Pahuja, AJ Piergiovanni, Anelia Angelova

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to Transformers for Vision (T4V) workshop, CVPR 2023; 7 pages, 5 figures

Scalable Neural-Probabilistic Answer Set Programming

Arseny Skryagin, Daniel Ochs, Devendra Singh Dhami, Kristian Kersting

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 37 pages, 14 figures

Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes

Alexandros Delitzas, Maria Parelli, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, Thomas Hofmann

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments The first two authors contributed equally. arXiv admin note: text overlap with arXiv:2304.06061

An Approach to Solving the Abstraction and Reasoning Corpus (ARC) Challenge

Tan John Chong Min

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 14 pages

PDFVQA: A New Dataset for Real-World VQA on PDF Documents

Yihao Ding, Siwen Luo, Hyunsuk Chung, Soyeon Caren Han

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by ECML-PKDD 2023

Evaluating the Capabilities of Multi-modal Reasoning Models with Synthetic Task Data

Nathan Vaska, Victoria Helus

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Multi-Scale Attention for Audio Question Answering

Guangyao Li, Yixin Xu, Di Hu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments Accepted by InterSpeech 2023

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Wei Zhan, Masayoshi Tomizuka, Mingyu Ding

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts

Qiuhui Chen, Xinyue Hu, Zirui Wang, Yi Hong

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 11 pages, 3 figures

CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes

Maria Parelli, Alexandros Delitzas, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, Thomas Hofmann

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPRW 2023. Code will be made publicly available: https://github.com/AlexDelitzas/3D-VQA

Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT

Mingzhe Hu, Shaoyan Pan, Yuheng Li, Xiaofeng Yang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

CLIPPO: Image-and-Language Understanding from Pixels Only

Michael Tschannen, Basil Mustafa, Neil Houlsby

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2023. Code and pretrained models are available at https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/clippo/README.md

Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning

Yatai Ji, Rongcheng Tu, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, Wei Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2023 accept

Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training

Anthony Meng Huat Tiong, Junnan Li, Boyang Li, Silvio Savarese, Steven C. H. Hoi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments EMNLP 2022 (Findings); correct typos in Equation 2 on page 4

Connecting Vision and Language with Video Localized Narratives

Paul Voigtlaender, Soravit Changpinyo, Jordi Pont-Tuset, Radu Soricut, Vittorio Ferrari

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

Comments Accepted at CVPR 2023

Linearly Mapping from Image to Text Space

Jack Merullo, Louis Castricato, Carsten Eickhoff, Ellie Pavlick

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Comments Accepted at ICLR 2023

Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts

Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, Xiang Wan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Work in progress

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Fernando Amodeo, Fernando Caballero, Natalia Díaz-Rodríguez, Luis Merino

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 20 pages; version accepted and published in IEEE Access

MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering

Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, Mike Zheng Shou

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV

3D Question Answering

Shuquan Ye, Dongdong Chen, Songfang Han, Jing Liao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To Appear at IEEE Transactions on Visualization and Computer Graphics (TVCG) 2022

Look, Read and Ask: Learning to Ask Questions by Reading Text in Images

Soumya Jahagirdar, Shankar Gangisetty, Anand Mishra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Cross-Modal Contrastive Learning for Robust Reasoning in VQA

Qi Zheng, Chaoyue Wang, Daqing Liu, Dadong Wang, Dacheng Tao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Continual VQA for Disaster Response Systems

Aditya Kane, V Manushree, Sahil Khose

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted at Tackling Climate Change with Machine Learning workshop at NeurIPS 2022

ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation

Bin Shan, Yaqian Han, Weichong Yin, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 13 pages, 2 figures

Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision

Tzu-Jui Julius Wang, Jorma Laaksonen, Tomas Langer, Heikki Arponen, Tom E. Bishop

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to WACV'23. Please find supplementary material at https://drive.google.com/file/d/1SmCBGsUgkYLAhmK83RZqY03bq4j3214p/view?usp=sharing

On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization

Shruti Palaskar, Akshita Bhagia, Yonatan Bisk, Florian Metze, Alan W Black, Ana Marasović

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments v2: EMNLP Findings 2022 accepted paper camera-ready version. 9 pages main, 2 pages appendix

Image Semantic Relation Generation

Mingzhe Du

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Fingdings of EMNLP-2022

RepsNet: Combining Vision with Language for Automated Medical Reports

Ajay Kumar Tanwani, Joelle Barral, Daniel Freedman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Journal ref MICCAI 2022, pp. 714--724