视觉大模型 / VLM

Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models

Zicheng Zhang, Haoning Wu, Zhongpeng Ji, Chunyi Li, Erli Zhang, Wei Sun, Xiaohong Liu, Xiongkuo Min, Fengyu Sun, Shangling Jui, Weisi Lin, Guangtao Zhai

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.CV

OsmLocator: locating overlapping scatter marks with a non-training generative perspective

Yuming Qiu, Aleksandra Pizurica, Qi Ming, Nicolas Nadisic

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 22pages

Reducing Hallucinations: Enhancing VQA for Flood Disaster Damage Assessment with Visual Contexts

Yimin Sun, Chao Wang, Yan Peng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments already be accepted by 2024 3rd International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE 2024)

Interactive Visual Task Learning for Robots

Weiwei Gu, Anant Sah, Nakul Gopalan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments In Proceedings of The 38th Annual AAAI Conference on Artificial Intelligence

VLIS: Unimodal Language Models Guide Multimodal Language Generation

Jiwan Chung, Youngjae Yu

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.AI

Comments Accepted as main paper in EMNLP 2023

VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation

Jinguo Zhu, Xiaohan Ding, Yixiao Ge, Yuying Ge, Sijie Zhao, Hengshuang Zhao, Xiaohua Wang, Ying Shan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Audio-Visual LLM for Video Understanding

Fangxun Shu, Lei Zhang, Hao Jiang, Cihang Xie

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models

Shitian Zhao, Zhuowan Li, Yadong Lu, Alan Yuille, Yan Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

How (not) to ensemble LVLMs for VQA

Lisa Alazraki, Lluis Castrejon, Mostafa Dehghani, Fantine Huot, Jasper Uijlings, Thomas Mensink

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments 4th I Can't Believe It's Not Better Workshop (co-located with NeurIPS 2023)

Global and Local Semantic Completion Learning for Vision-Language Pre-training

Rong-Cheng Tu, Yatai Ji, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, Wei Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2211.13437

Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data

Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding

Junyan Li, Delin Chen, Yining Hong, Zhenfang Chen, Peihao Chen, Yikang Shen, Chuang Gan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Adaptive loose optimization for robust question answering

Jie Ma, Pinghui Wang, Zewei Wang, Dechen Kong, Min Hu, Ting Han, Jun Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 13 pages,8 figures

Reusable Slotwise Mechanisms

Trang Nguyen, Amin Mansouri, Kanika Madan, Khuong Nguyen, Kartik Ahuja, Dianbo Liu, Yoshua Bengio

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Jaeger: A Concatenation-Based Multi-Transformer VQA Model

Jieting Long, Zewei Shi, Penghao Jiang, Yidong Gan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments This paper is the technical research paper of CIKM 2023 DocIU challenges. The authors received the CIKM 2023 DocIU Winner Award, sponsored by Google, Microsoft, and the Centre for data-driven geoscience

A scoping review on multimodal deep learning in biomedical images and texts

Zhaoyi Sun, Mingquan Lin, Qingqing Zhu, Qianqian Xie, Fei Wang, Zhiyong Lu, Yifan Peng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments This paper has been accepted by the Journal of Biomedical Informatics

Journal ref Journal of Biomedical Informatics, Volume 146, October 2023, 104482

Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA

Sheng Zhou, Dan Guo, Jia Li, Xun Yang, Meng Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by TIP 2023

Human Mobility Question Answering (Vision Paper)

Hao Xue, Flora D. Salim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering

Bruno Souza, Marius Aasan, Helio Pedrini, Adín Ramírez Rivera

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To appear in Vision-and-Language Algorithmic Reasoning Workshop at ICCV 2023

Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts

Deniz Engin, Yannis Avrithis

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments ICCV2023 CLVL Workshop (Oral). Project page: https://engindeniz.github.io/vitis

Tackling VQA with Pretrained Foundation Models without Further Training

Alvin De Jun Tan, Bingquan Shen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Pointing out Human Answer Mistakes in a Goal-Oriented Visual Dialogue

Ryosuke Oshima, Seitaro Shinagawa, Hideki Tsunashima, Qi Feng, Shigeo Morishima

专题命中 视觉问答 ：visual language model(abstract)；分类 cs.CV

Comments Accepted at ICCVW 2023

Towards Language-guided Visual Recognition via Dynamic Convolutions

Gen Luo, Yiyi Zhou, Xiaoshuai Sun, Yongjian Wu, Yue Gao, Rongrong Ji

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Generic Attention-model Explainability by Weighted Relevance Accumulation

Yiming Huang, Aozhe Jia, Xiaodan Zhang, Jiawei Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision

Sophia Gu, Christopher Clark, Aniruddha Kembhavi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments website (https://prior.allenai.org/projects/close), code (https://github.com/allenai/close)

PreSTU: Pre-Training for Scene-Text Understanding

Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei-Lun Chao, Radu Soricut

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, Noah A Smith

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

PromptCap: Prompt-Guided Task-Aware Image Captioning

Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, Jiebo Luo

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models

Indranil Sur, Karan Sikka, Matthew Walmer, Kaushik Koneripalli, Anirban Roy, Xiao Lin, Ajay Divakaran, Susmit Jha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Published as conference paper at ICCV 2023. 13 pages, 6 figures, 7 tables

Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories

Thomas Mensink, Jasper Uijlings, Lluis Castrejon, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, André Araujo, Vittorio Ferrari

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ICCV'23