视觉大模型 / VLM - arXivDaily 专题

2501.05452 2025-01-10 cs.CV cs.CL 70%

ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding

Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, Cha Zhang

专题命中视觉问答：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Project link: https://zeyofu.github.io/ReFocus/

2412.17787 2024-12-24 cs.CV cs.CL 70%

Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective

Xinmiao Yu, Xiaocheng Feng, Yun Li, Minghui Liao, Ya-Qi Yu, Xiachong Feng, Weihong Zhong, Ruihan Chen, Mengkang Hu, Jihao Wu, Dandan Tu, Duyu Tang, Bing Qin

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

2412.16420 2024-12-24 cs.CV cs.CL 70%

Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understanding

Junyi Ye, Ankan Dash, Wenpeng Yin, Guiling Wang

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2412.08859 2024-12-13 cs.CV 70%

ViUniT: Visual Unit Tests for More Robust Visual Programming

Artemis Panagopoulou, Honglu Zhou, Silvio Savarese, Caiming Xiong, Chris Callison-Burch, Mark Yatskar, Juan Carlos Niebles

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

2410.10799 2024-12-10 cs.CV 70%

Towards Foundation Models for 3D Vision: How Close Are We?

Yiming Zuo, Karhan Kayan, Maggie Wang, Kevin Jeon, Jia Deng, Thomas L. Griffiths

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to 3DV 2025. Update 12/09/24: Change the benchmark name to UniQA-3D, add link to code

2411.10669 2024-11-19 cs.CV 70%

Awaker2.5-VL: Stably Scaling MLLMs with Parameter-Efficient Mixture of Experts

Jinqiang Long, Yanqi Dai, Guoxing Yang, Hongpeng Lin, Nanyi Fei, Yizhao Gao, Zhiwu Lu

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2411.04006 2024-11-07 cs.RO cs.AI 70%

Select2Plan: Training-Free ICL-Based Planning through VQA and Memory Retrieval

Davide Buoso, Luke Robinson, Giuseppe Averta, Philip Torr, Tim Franzmeyer, Daniele De Martini

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

2410.22648 2024-10-31 cs.CV 70%

SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset

Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Sunil Aryal, Imran Razzak, Hakim Hacid

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

2410.22029 2024-10-30 cs.CL cs.CV 70%

Are VLMs Really Blind

Ayush Singh, Mansi Gupta, Shivank Garg

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 2 pages, 1 figure

2408.03361 2024-10-22 eess.IV cs.CV 70%

GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI

Pengcheng Chen, Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, Wei Li, Tianbin Li, Haodong Duan, Ziyan Huang, Yanzhou Su, Benyou Wang, Shaoting Zhang, Bin Fu, Jianfei Cai, Bohan Zhuang, Eric J Seibel, Junjun He, Yu Qiao

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments GitHub: https://github.com/uni-medical/GMAI-MMBench Hugging face: https://huggingface.co/datasets/OpenGVLab/GMAI-MMBench

2410.12109 2024-10-17 cs.CL cs.CV 70%

OMCAT: Omni Context Aware Transformer

Arushi Goel, Karan Sapra, Matthieu Le, Rafael Valle, Andrew Tao, Bryan Catanzaro

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Demo page: https://om-cat.github.io

2410.11437 2024-10-16 cs.CL cs.AI 70%

Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs

Sihang Zhao, Youliang Yuan, Xiaoying Tang, Pinjia He

专题命中视觉问答：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.AI

Comments EMNLP 2024 Findings

2409.09582 2024-09-25 cs.CV 70%

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

Yiyi Tao, Zhuoyue Wang, Hang Zhang, Lun Wang

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

2403.03174 2024-09-05 cs.RO cs.AI 70%

MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting

Fangchen Liu, Kuan Fang, Pieter Abbeel, Sergey Levine

专题命中视觉问答：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

2403.16921 2024-07-24 cs.CV 70%

PropTest: Automatic Property Testing for Improved Visual Programming

Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Project Page: https://jaywonkoo17.github.io/PropTest/

2402.13220 2024-07-24 cs.CV cs.CL 70%

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts

Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan

专题命中视觉问答：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

2406.16817 2024-06-25 cs.CV 70%

GPT-4V Explorations: Mining Autonomous Driving

Zixuan Li

专题命中视觉问答：visual language model(abstract)；visual question answering(abstract)；分类 cs.CV

2402.00453 2024-06-14 cs.CV cs.CL 70%

Instruction Makes a Difference

Tosin Adewumi, Nudrat Habib, Lama Alkhaled, Elisa Barney

专题命中视觉问答：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted at the 16th IAPR International Workshop On Document Analysis Systems (DAS)

2406.04236 2024-06-07 cs.CV 70%

Understanding Information Storage and Transfer in Multi-modal Large Language Models

Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, Daniela Massiceti

专题命中视觉问答：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 20 pages

2405.16886 2024-05-28 cs.CV 70%

Hawk: Learning to Understand Open-World Video Anomalies

Jiaqi Tang, Hao Lu, Ruizheng Wu, Xiaogang Xu, Ke Ma, Cheng Fang, Bin Guo, Jiangbo Lu, Qifeng Chen, Ying-Cong Chen

专题命中视觉问答：VLM(abstract)；visual language model(abstract)；分类 cs.CV

2402.09181 2024-04-23 eess.IV cs.CV 70%

OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM

Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, Ping Luo

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

2312.03587 2024-04-04 cs.CV 70%

Language-Informed Visual Concept Learning

Sharon Lee, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments ICLR 2024. The first two authors contributed equally and are alphabetically ordered. Project page: https://ai.stanford.edu/~yzzhang/projects/concept-axes/

2403.15456 2024-04-01 cs.AI cs.CL 70%

WoLF: Wide-scope Large Language Model Framework for CXR Understanding

Seil Kang, Donghyun Kim, Junhyeok Kim, Hyo Kyung Lee, Seong Jae Hwang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.AI

Comments 11 pages main paper, 2 pages supplementary

2310.08825 2024-03-11 cs.CV 70%

From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Dongsheng Jiang, Yuchen Liu, Songlin Liu, Jin'e Zhao, Hao Zhang, Zhen Gao, Xiaopeng Zhang, Jin Li, Hongkai Xiong

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2302.00389 2024-03-04 cs.AI 70%

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, Shangsong Liang

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.AI

2402.05138 2024-02-09 cs.AI cs.CL 70%

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang, Jiajun Jiao, Renjie Pi, Jipeng Zhang, Xiangliang Zhang

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Work in progress

2312.14074 2023-12-22 cs.CV 70%

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

Senqiao Yang, Jiaming Liu, Ray Zhang, Mingjie Pan, Zoey Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Yandong Guo, Shanghang Zhang

专题命中视觉问答：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2312.06352 2023-12-12 cs.CV cs.CL 70%

NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations

Yuichi Inoue, Yuki Yada, Kotaro Tanahashi, Yu Yamaguchi

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted at LLVM-AD Workshop @ WACV 2024

2305.12397 2023-12-11 cs.CV 70%

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

Yuanyuan Jiang, Jianqin Yin

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted to EMNLP 2023 Findings

2311.17043 2023-11-29 cs.CV cs.CL 70%

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

Yanwei Li, Chengyao Wang, Jiaya Jia

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Code is available at https://github.com/dvlab-research/LLaMA-VID

1. 视觉问答 3126 篇

ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding

Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective

Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understanding

ViUniT: Visual Unit Tests for More Robust Visual Programming

Towards Foundation Models for 3D Vision: How Close Are We?

Awaker2.5-VL: Stably Scaling MLLMs with Parameter-Efficient Mixture of Experts

Select2Plan: Training-Free ICL-Based Planning through VQA and Memory Retrieval

SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset

Are VLMs Really Blind

GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI

OMCAT: Omni Context Aware Transformer

Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting

PropTest: Automatic Property Testing for Improved Visual Programming

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts

GPT-4V Explorations: Mining Autonomous Driving

Instruction Makes a Difference

Understanding Information Storage and Transfer in Multi-modal Large Language Models

Hawk: Learning to Understand Open-World Video Anomalies

OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM

Language-Informed Visual Concept Learning

WoLF: Wide-scope Large Language Model Framework for CXR Understanding

From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models