视觉大模型 / VLM

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, Yu Qiao

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.CV

Comments CVPR 2024 highlight: updated version with Mistral and better performances for MVBench/NExT-QA/STAR/TVQA/EgoSchema/IntentQA

Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding

Zheng Zhao, Emilio Monti, Jens Lehmann, Haytham Assem

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

Comments Accepted to NAACL 2024

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, Gaoang Wang

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

MM-PhyQA: Multimodal Physics Question-Answering With Multi-Image CoT Prompting

Avinash Anand, Janak Kapuriya, Apoorv Singh, Jay Saraf, Naman Lal, Astha Verma, Rushali Gupta, Rajiv Shah

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.AI

CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update

Zhi Gao, Yuntao Du, Xintong Zhang, Xiaojian Ma, Wenjuan Han, Song-Chun Zhu, Qing Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2024

Mitigating the Impact of Attribute Editing on Face Recognition

Sudipta Banerjee, Sai Pranaswi Mullangi, Shruti Wagle, Chinmay Hegde, Nasir Memon

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV

Comments Under review

Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning

Juncheng Yang, Zuchao Li, Shuai Xie, Wei Yu, Shijun Li, Bo Du

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments This paper is accepted to LREC-COLING 2024

Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement

Zaid Khan, Vijay Kumar BG, Samuel Schulter, Yun Fu, Manmohan Chandraker

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2024

Can LLMs Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesis

Vishnu Sashank Dorbala, Sanjoy Chowdhury, Dinesh Manocha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 14 Pages

Grounded Question-Answering in Long Egocentric Videos

Shangzhe Di, Weidi Xie

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

Comments Accepted to CVPR 2024. Project website at https://dszdsz.cn/GroundVQA

Visual Superordinate Abstraction for Robust Concept Learning

Qi Zheng, Chaoyue Wang, Dadong Wang, Dacheng Tao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 14 pages, 11 figures

Journal ref Machine Intelligence Research vol. 20, no. 1, pp. 79-91, 2024

Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA

Zhuowan Li, Bhavan Jasani, Peng Tang, Shabnam Ghadar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to CVPR 2024

CLAMP: Contrastive LAnguage Model Prompt-tuning

Piotr Teterwak, Ximeng Sun, Bryan A. Plummer, Kate Saenko, Ser-Nam Lim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

ViT-Lens: Initiating Omni-Modal Exploration through 3D Insights

Weixian Lei, Yixiao Ge, Jianfeng Zhang, Dylan Sun, Kun Yi, Ying Shan, Mike Zheng Shou

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments 19 pages, 4 figures and 9 tables

Improved Baselines for Data-efficient Perceptual Augmentation of LLMs

Théophane Vallaeys, Mustafa Shukor, Matthieu Cord, Jakob Verbeek

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Few-Shot VQA with Frozen LLMs: A Tale of Two Approaches

Igor Sterner, Weizhe Lin, Jinghong Chen, Bill Byrne

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

GRAM: Global Reasoning for Multi-Page VQA

Tsachi Blau, Sharon Fogel, Roi Ronen, Alona Golts, Roy Ganz, Elad Ben Avraham, Aviad Aberdam, Shahar Tsiper, Ron Litman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Knowledge Condensation and Reasoning for Knowledge-based VQA

Dongze Hao, Jian Jia, Longteng Guo, Qunbo Wang, Te Yang, Yan Li, Yanhua Cheng, Bo Wang, Quan Chen, Han Li, Jing Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework

Chris Kelly, Luhui Hu, Bang Yang, Yu Tian, Deshun Yang, Cindy Yang, Zaoshan Huang, Zihao Li, Jiayin Hu, Yuexian Zou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 17 pages, 5 figures, and 1 table. arXiv admin note: substantial text overlap with arXiv:2311.10125

Beyond Text: Frozen Large Language Models in Visual Signal Comprehension

Lei Zhu, Fangyun Wei, Yanye Lu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Answering Diverse Questions via Text Attached with Key Audio-Visual Clues

Qilang Ye, Zitong Yu, Xin Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training

Alyssa Huang, Peihan Liu, Ryumei Nakada, Linjun Zhang, Wanrong Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Training Table Question Answering via SQL Query Decomposition

Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

KVQ: Kwai Video Quality Assessment for Short-form Videos

Yiting Lu, Xin Li, Yajing Pei, Kun Yuan, Qizhi Xie, Yunpeng Qu, Ming Sun, Chao Zhou, Zhibo Chen

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV

Comments 19 pages

Modularized Zero-shot VQA with Pre-trained Models

Rui Cao, Jing Jiang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments accepted as Findings in ACL 2023; Code available: https://github.com/abril4416/Mod-Zero-VQA

Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation

Songhe Deng, Wei Zhuo, Jinheng Xie, Linlin Shen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ACM MM 2023

Gemini Pro Defeated by GPT-4V: Evidence from Education

Gyeong-Geon Lee, Ehsan Latif, Lehong Shi, Xiaoming Zhai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

NAAQA: A Neural Architecture for Acoustic Question Answering

Jerome Abdelnour, Jean Rouat, Giampiero Salvi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Comments Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) in April 2021 (first revision February 2022)

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, Volume: 45 Issue: 4, Page(s): 4997-5009

Glance and Focus: Memory Prompting for Multi-Event Video Question Answering

Ziyi Bai, Ruiping Wang, Xilin Chen

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV

Comments Accepted in NeurIPS 2023

Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models

Xinpeng Ding, Jinahua Han, Hang Xu, Xiaodan Liang, Wei Zhang, Xiaomeng Li

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV