视觉大模型 / VLM

Multimodal Hypothetical Summary for Retrieval-based Multi-image Question Answering

Peize Li, Qingyi Si, Peng Fu, Zheng Lin, Yan Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

Comments AAAI 2025

MedCoT: Medical Chain of Thought via Hierarchical Expert

Jiaxiang Liu, Yuan Wang, Jiawei Du, Joey Tianyi Zhou, Zuozhu Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Journal ref EMNLP 2024

Track the Answer: Extending TextVQA from Image to Video with Spatio-Temporal Clues

Yan Zhang, Gangyan Zeng, Huawen Shen, Daiqing Wu, Yu Zhou, Can Ma

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by AAAI 2025

CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology

Yuxuan Sun, Yixuan Si, Chenglu Zhu, Xuan Gong, Kai Zhang, Pingyi Chen, Ye Zhang, Zhongyi Shui, Tao Lin, Lin Yang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 22 pages, 13 figures

CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding

Guo Chen, Yicheng Liu, Yifei Huang, Yuping He, Baoqi Pei, Jilan Xu, Yali Wang, Tong Lu, Limin Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

Comments 14 pages, 9 figures

ACE-$M^3$: Automatic Capability Evaluator for Multimodal Medical Models

Xiechi Zhang, Shunfan Zheng, Linlin Wang, Gerard de Melo, Zhu Cao, Xiaoling Wang, Liang He

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.AI

Overview of TREC 2024 Medical Video Question Answering (MedVidQA) Track

Deepak Gupta, Dina Demner-Fushman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusions

Mohammadmostafa Rostamkhani, Baktash Ansari, Hoorieh Sabzevari, Farzan Rahmani, Sauleh Eetemadi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges

Minghao Shao, Abdul Basit, Ramesh Karri, Muhammad Shafique

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.LG

Eyes on the Road: State-of-the-Art Video Question Answering Models Assessment for Traffic Monitoring Tasks

Joseph Raj Vishal, Divesh Basina, Aarya Choudhary, Bharatesh Chakravarthi

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

EchoSight: Advancing Visual-Language Models with Wiki Knowledge

Yibin Yan, Weidi Xie

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by EMNLP 2024 findings; Project Page: https://go2heart.github.io/echosight

A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models

Jie Liu, Wenxuan Wang, Yihang Su, Jingyuan Huan, Wenting Chen, Yudi Zhang, Cheng-Yi Li, Kao-Jung Chang, Xiaohan Xin, Linlin Shen, Michael R. Lyu

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.CV

Comments 20 pages, 15 figures

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation

Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, Chao Zhang

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.AI

Comments Technical report

A Survey of Medical Vision-and-Language Applications and Their Techniques

Qi Chen, Ruoshan Zhao, Sinuo Wang, Vu Minh Hieu Phan, Anton van den Hengel, Johan Verjans, Zhibin Liao, Minh-Son To, Yong Xia, Jian Chen, Yutong Xie, Qi Wu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

PQA: Zero-shot Protein Question Answering for Free-form Scientific Enquiry with Large Language Models

Eli M Carrami, Sahand Sharifzadeh

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.LG

Memory-Augmented Multimodal LLMs for Surgical VQA via Self-Contained Inquiry

Wenjun Hou, Yi Cheng, Kaishuai Xu, Yan Hu, Wenjie Li, Jiang Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Efficient Large Multi-modal Models via Visual Context Compression

Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, Alan Yuille

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments NeurIPS 2024 Camera Ready; Code is available at https://github.com/Beckschen/LLaVolta

SparrowVQE: Visual Question Explanation for Course Content Understanding

Jialu Li, Manish Kumar Thota, Ruslan Gokhman, Radek Holik, Youshan Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Poze: Sports Technique Feedback under Data Constraints

Agamdeep Singh, Sujit PB, Mayank Vatsa

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Models

Pete Janowczyk, Linda Laurier, Ave Giulietta, Arlo Octavia, Meade Cleti

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

MME-Finance: A Multimodal Finance Benchmark for Expert-level Understanding and Reasoning

Ziliang Gan, Yu Lu, Dong Zhang, Haohan Li, Che Liu, Jian Liu, Ji Liu, Haipang Wu, Chaoyou Fu, Zenglin Xu, Rongjunchen Zhang, Yong Dai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Project Page: https://hithink-research.github.io/MME-Finance/

A Hitchhikers Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoning

Niki Maria Foteinopoulou, Enjie Ghorbel, Djamila Aouada

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted at NeurIPS'2024 (D&B)

Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications

Monica Riedler, Stefan Langer

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.AI

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Yonggang Wen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

SemiHVision: Enhancing Medical Multimodal Models with a Semi-Human Annotated Dataset and Fine-Tuned Instruction Generation

Junda Wang, Yujan Ting, Eric Z. Chen, Hieu Tran, Hong Yu, Weijing Huang, Terrence Chen

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training

Muhe Ding, Yang Ma, Pengda Qin, Jianlong Wu, Yuhong Li, Liqiang Nie

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.AI

Comments 10 pages, 6 figures, Journal

Help Me Identify: Is an LLM+VQA System All We Need to Identify Visual Concepts?

Shailaja Keyur Sampat, Maitreya Patel, Yezhou Yang, Chitta Baral

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 14 pages, 7 figures

NatLan: Native Language Prompting Facilitates Knowledge Elicitation Through Language Trigger Provision and Domain Trigger Retention

Baixuan Li, Yunlong Fan, Tianyi Ma, Zhiqiang Gao

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.AI

TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy

Weichao Zhao, Hao Feng, Qi Liu, Jingqun Tang, Shu Wei, Binghong Wu, Lei Liao, Yongjie Ye, Hao Liu, Wengang Zhou, Houqiang Li, Can Huang

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning

Thong Nguyen, Yi Bin, Xiaobao Wu, Xinshuai Dong, Zhiyuan Hu, Khoi Le, Cong-Duy Nguyen, See-Kiong Ng, Luu Anh Tuan

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024