视觉大模型 / VLM

Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering

He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perception

Junxiao Shen, John Dudley, Per Ola Kristensson

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV、cs.AI

HAMMR: HierArchical MultiModal React agents for generic VQA

Lluis Castrejon, Thomas Mensink, Howard Zhou, Vittorio Ferrari, Andre Araujo, Jasper Uijlings

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Haibo Wang, Weifeng Ge

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV'24

Core Tokensets for Data-efficient Sequential Training of Transformers

Subarnaduti Paul, Manuel Brack, Patrick Schramowski, Kristian Kersting, Martin Mundt

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Attribute Diversity Determines the Systematicity Gap in VQA

Ian Berlot-Attwell, Kumar Krishna Agrawal, A. Michael Carrell, Yash Sharma, Naomi Saphra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 36 pages, 27 figures, EMNLP 2024

A comprehensive study of on-device NLP applications -- VQA, automated Form filling, Smart Replies for Linguistic Codeswitching

Naman Goyal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Models

Saman Motamed, Danda Pani Paudel, Luc Van Gool

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Explore the Hallucination on Low-level Perception for MLLMs

Yinan Sun, Zicheng Zhang, Haoning Wu, Xiaohong Liu, Weisi Lin, Guangtao Zhai, Xiongkuo Min

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Top-down Activity Representation Learning for Video Question Answering

Yanan Wang, Shuichiro Haruta, Donghuo Zeng, Julio Vizcarra, Mori Kurokawa

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments presented at MIRU2024

EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysis

Danli Shi, Weiyi Zhang, Jiancheng Yang, Siyu Huang, Xiaolan Chen, Mayinuer Yusufu, Kai Jin, Shan Lin, Shunming Liu, Qing Zhang, Mingguang He

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models

Bin Fu, Qiyang Wan, Jialin Li, Ruiping Wang, Xilin Chen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 39 pages, 28 figures, 4 tables. Accepted at The 35th British Machine Vision Conference (BMVC 2024). Project page at https://fubin29.github.io/Blocks-as-Probes/

Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models

Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, Xiang Bai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024 Highlight

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, Jae Sung Park, Jack Hessel, Lijuan Wang, Yejin Choi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 26 pages

Advancing Surgical VQA with Scene Graph Knowledge

Kun Yuan, Manasi Kattel, Joel L. Lavanchy, Nassir Navab, Vinkle Srivastav, Nicolas Padoy

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments IPCAI 2024, Int J CARS (2024)

MISS: A Generative Pretraining and Finetuning Approach for Med-VQA

Jiawei Chen, Dingkang Yang, Yue Jiang, Yuxuan Lei, Lihua Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments ICANN, 2024

KNVQA: A Benchmark for evaluation knowledge-based VQA

Sirui Cheng, Siyu Zhang, Jiayi Wu, Muchen Lan

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments There was a little error in the method section of the paper

Lumos : Empowering Multimodal LLMs with Scene Text Recognition

Ashish Shenoy, Yichao Lu, Srihari Jayakumar, Debojeet Chatterjee, Mohsen Moslehpour, Pierce Chuang, Abhay Harpale, Vikas Bhardwaj, Di Xu, Shicong Zhao, Longfang Zhao, Ankit Ramchandani, Xin Luna Dong, Anuj Kumar

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to KDD 2024 (ADS Track)

Retrieval Augmented Generation for Domain-specific Question Answering

Sanat Sharma, David Seunghyun Yoon, Franck Dernoncourt, Dewang Sultania, Karishma Bagga, Mengjiao Zhang, Trung Bui, Varun Kotte

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments AAAI 2024 (Association for the Advancement of Artificial Intelligence) Scientific Document Understanding Workshop

EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imaging

Danli Shi, Weiyi Zhang, Xiaolan Chen, Yexin Liu, Jiancheng Yang, Siyu Huang, Yih Chung Tham, Yingfeng Zheng, Mingguang He

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 21 pages, 2 figures, 4 tables

Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers

Georgios Pantazopoulos, Alessandro Suglia, Oliver Lemon, Arash Eshghi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments NAACL 2024

Graph Neural Networks in Vision-Language Image Understanding: A Survey

Henry Senior, Gregory Slabaugh, Shanxin Yuan, Luca Rossi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 20 pages, 5 figures, 5 tables

Journal ref Vis Comput (2024)

FunQA: Towards Surprising Video Comprehension

Binzhu Xie, Sicheng Zhang, Zitang Zhou, Bo Li, Yuanhan Zhang, Jack Hessel, Jingkang Yang, Ziwei Liu

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://funqa-benchmark.github.io/ Codebase: https://github.com/Jingkang50/FunQA

Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use

Imad Eddine Toubal, Aditya Avinash, Neil Gordon Alldrin, Jan Dlabal, Wenlei Zhou, Enming Luo, Otilia Stretcu, Hao Xiong, Chun-Ta Lu, Howard Zhou, Ranjay Krishna, Ariel Fuxman, Tom Duerig

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Evaluating Very Long-Term Conversational Memory of LLM Agents

Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, Yuwei Fang

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments 19 pages; Project page: https://snap-research.github.io/locomo/

Multimodal Transformer With a Low-Computational-Cost Guarantee

Sungjin Park, Edward Choi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ICASSP 2024 (5 pages)

MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing

Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong Tian

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments 8 pages

An Evaluation of GPT-4V and Gemini in Online VQA

Mengchen Liu, Chongyan Chen, Danna Gurari

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 20 pages

Knowledge Generation for Zero-shot Knowledge-based VQA

Rui Cao, Jing Jiang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments accepted as Findings in EACL 2023

On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications

Chenjiao Tan, Qian Cao, Yiwei Li, Jielu Zhang, Xiao Yang, Huaqin Zhao, Zihao Wu, Zhengliang Liu, Hao Yang, Nemin Wu, Tao Tang, Xinyue Ye, Lilong Chai, Ninghao Liu, Changying Li, Lan Mu, Tianming Liu, Gengchen Mai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 110 Pages; 61 Figures