视觉大模型 / VLM - arXivDaily 专题

2406.01584 2024-10-16 cs.CV 79%

SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models

An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, Sifei Liu

专题命中视觉推理：vision language model(title,abstract)；分类 cs.CV

Comments NeurIPS 2024, Project Page: https://www.anjiecheng.me/SpatialRGPT

2408.12114 2024-10-14 cs.CV 79%

SPARK: Multi-Vision Sensor Perception and Reasoning Benchmark for Large-scale Vision-Language Models

Youngjoon Yu, Sangyun Chung, Byung-Kwan Lee, Yong Man Ro

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

Comments Codes and data are available at https://github.com/top-yun/SPARK

2407.15272 2024-10-10 cs.CV 79%

MIBench: Evaluating Multimodal Large Language Models over Multiple Images

Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, Weiming Hu

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.CV

Comments EMNLP 2024

2405.14156 2024-10-08 cs.CV 79%

Unveiling the Tapestry of Consistency in Large Vision-Language Models

Yuan Zhang, Fei Xiao, Tao Huang, Chun-Kai Fan, Hongyuan Dong, Jiawen Li, Jiacong Wang, Kuan Cheng, Shanghang Zhang, Haoyuan Guo

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2407.07053 2024-10-04 cs.CV 79%

Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model

Wenqi Zhang, Zhenglin Cheng, Yuanyu He, Mengna Wang, Yongliang Shen, Zeqi Tan, Guiyang Hou, Mingqian He, Yanna Ma, Weiming Lu, Yueting Zhuang

专题命中视觉推理：visual reasoning(title,abstract)；分类 cs.CV

Comments The paper is accepted by EMNLP-24. Code: https://github.com/zwq2018/Multi-modal-Self-instruct dataset: https://huggingface.co/datasets/zwq2018/Multi-modal-Self-instruct Leaderboard: https://multi-modal-self-instruct.github.io/

2404.09486 2024-09-27 cs.CL cs.CV cs.SE 79%

MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems

Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, Zhiyong Huang, Jing Ma

专题命中视觉推理：multimodal large language model(title)；visual reasoning(abstract)；分类 cs.CV

Comments EMNLP 2024

2409.06450 2024-09-11 cs.RO cs.AI cs.ET 79%

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Qiujing Lu, Xuanhan Wang, Yiwei Jiang, Guangming Zhao, Mingyue Ma, Shuo Feng

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.AI

2408.12321 2024-08-27 cs.CL cs.CV cs.MM 79%

MaVEn: An Effective Multi-granularity Hybrid Visual Encoding Framework for Multimodal Large Language Model

Chaoya Jiang, Jia Hongrui, Haiyang Xu, Wei Ye, Mengfan Dong, Ming Yan, Ji Zhang, Fei Huang, Shikun Zhang

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.CV

2407.17379 2024-08-07 cs.CV cs.CL 79%

MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models

Siwei Wu, Kang Zhu, Yu Bai, Yiming Liang, Yizhi Li, Haoning Wu, J. H. Liu, Ruibo Liu, Xingwei Qu, Xuxin Cheng, Ge Zhang, Wenhao Huang, Chenghua Lin

专题命中视觉推理：visual language model(title,abstract)；分类 cs.CV

Comments VLMs, Multi-Image Association

2408.02210 2024-08-06 cs.CV 79%

ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning

Yuxuan Wang, Alan Yuille, Zhuowan Li, Zilong Zheng

专题命中视觉推理：visual reasoning(title,abstract)；分类 cs.CV

Comments To Appear at COLM 2024

2407.14834 2024-07-23 cs.CV 79%

Can VLMs be used on videos for action recognition? LLMs are Visual Reasoning Coordinators

Harsh Lunia

专题命中视觉推理：visual reasoning(title)；vision-language model(abstract)；分类 cs.CV

Comments LLMs, VLMs, Action Recognition

2406.10157 2024-07-23 cs.RO cs.AI 79%

RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model

Hantao Zhou, Tianying Ji, Lukas Sommerhalder, Michael Goerner, Norman Hendrich, Jianwei Zhang, Fuchun Sun, Huazhe Xu

专题命中视觉推理：vision-language model(title)；VLM(abstract)；分类 cs.AI

Comments Project page: https://jity16.github.io/RoboGolf/

2406.16449 2024-07-19 cs.CV 79%

Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models

Mingrui Wu, Jiayi Ji, Oucheng Huang, Jiale Li, Yuhang Wu, Xiaoshuai Sun, Rongrong Ji

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

Comments ICML2024; Project Page:https://github.com/mrwu-mac/R-Bench

2404.12065 2024-07-15 cs.CL cs.AI cs.CY cs.ET cs.MA 79%

RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models

M. Abdul Khaliq, P. Chang, M. Ma, B. Pflugfelder, F. Miletić

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.AI

Comments 8 pages, submitted to ACL Rolling Review June 2024

2403.10854 2024-07-12 cs.CV 79%

A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment

Tianhe Wu, Kede Ma, Jie Liang, Yujiu Yang, Lei Zhang

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.CV

2406.19131 2024-06-28 cs.CV 79%

CELLO: Causal Evaluation of Large Vision-Language Models

Meiqi Chen, Bo Peng, Yan Zhang, Chaochao Lu

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

2402.12289 2024-06-26 cs.CV 79%

DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models

Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, Hang Zhao

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

Comments Project Page: https://tsinghua-mars-lab.github.io/DriveVLM/

2312.06722 2024-06-12 cs.CV cs.CL cs.RO 79%

EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning

Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, Xihui Liu

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.CV

Comments Project released at: https://github.com/ChenYi99/EgoPlan

2405.17201 2024-05-28 cs.CV 79%

Diagnosing the Compositional Knowledge of Vision Language Models from a Game-Theoretic View

Jin Wang, Shichao Dong, Yapeng Zhu, Kelu Yao, Weidong Zhao, Chao Li, Ping Luo

专题命中视觉推理：vision language model(title,abstract)；分类 cs.CV

Comments 21 pages, 8 figures

2405.13751 2024-05-24 cs.RO cs.AI 79%

GameVLM: A Decision-making Framework for Robotic Task Planning Based on Visual Language Models and Zero-sum Games

Aoran Mei, Jianhua Wang, Guo-Niu Zhu, Zhongxue Gan

专题命中视觉推理：visual language model(title)；VLM(abstract)；分类 cs.AI

2404.00906 2024-04-25 cs.CV 79%

From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models

Rongjie Li, Songyang Zhang, Dahua Lin, Kai Chen, Xuming He

专题命中视觉推理：vision-language model(title)；VLM(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2404.07031 2024-04-11 cs.CV 79%

ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling

Ege Özsoy, Chantal Pellegrini, Matthias Keicher, Nassir Navab

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

Comments 11 pages, 3 figures, 7 tables

2403.12801 2024-03-20 cs.CV 79%

RelationVLM: Making Large Vision-Language Models Understand Visual Relations

Zhipeng Huang, Zhizheng Zhang, Zheng-Jun Zha, Yan Lu, Baining Guo

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

2403.10378 2024-03-18 cs.CL cs.CV 79%

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models

Rocktim Jyoti Das, Simeon Emilov Hristov, Haonan Li, Dimitar Iliyanov Dimitrov, Ivan Koychev, Preslav Nakov

专题命中视觉推理：vision language model(title,abstract)；分类 cs.CV

2403.06059 2024-03-12 cs.CV 79%

Test-time Distribution Learning Adapter for Cross-modal Visual Reasoning

Yi Zhang, Ce Zhang

专题命中视觉推理：visual reasoning(title,abstract)；分类 cs.CV

Comments Accepted by ICASSP 2024

2403.02330 2024-03-05 cs.CV 79%

RegionGPT: Towards Region Understanding Vision Language Model

Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, Sifei Liu

专题命中视觉推理：vision language model(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2402.02205 2024-02-08 cs.CV 79%

GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Events

Xingcheng Zhou, Alois C. Knoll

专题命中视觉推理：vision language model(title)；vision-language model(abstract)；分类 cs.CV

2311.14580 2023-11-27 cs.CV 79%

Large Language Models as Automated Aligners for benchmarking Vision-Language Models

Yuanfeng Ji, Chongjian Ge, Weikai Kong, Enze Xie, Zhengying Liu, Zhengguo Li, Ping Luo

专题命中视觉推理：vision-language model(title,abstract)；分类 cs.CV

2306.02500 2023-11-14 cs.CV 79%

Systematic Visual Reasoning through Object-Centric Relational Abstraction

Taylor W. Webb, Shanka Subhra Mondal, Jonathan D. Cohen

专题命中视觉推理：visual reasoning(title,abstract)；分类 cs.CV

2310.13473 2023-10-23 cs.CV 79%

Benchmarking Sequential Visual Input Reasoning and Prediction in Multimodal Large Language Models

Mingwei Zhu, Leigang Sha, Yu Shu, Kangjia Zhao, Tiancheng Zhao, Jianwei Yin

专题命中视觉推理：multimodal large language model(title,abstract)；分类 cs.CV

1. 视觉推理 4454 篇

SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models

SPARK: Multi-Vision Sensor Perception and Reasoning Benchmark for Large-scale Vision-Language Models

MIBench: Evaluating Multimodal Large Language Models over Multiple Images

Unveiling the Tapestry of Consistency in Large Vision-Language Models

Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model

MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

MaVEn: An Effective Multi-granularity Hybrid Visual Encoding Framework for Multimodal Large Language Model

MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models

ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning

Can VLMs be used on videos for action recognition? LLMs are Visual Reasoning Coordinators

RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model

Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models

RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models

A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment

CELLO: Causal Evaluation of Large Vision-Language Models

DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models

EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning

Diagnosing the Compositional Knowledge of Vision Language Models from a Game-Theoretic View

GameVLM: A Decision-making Framework for Robotic Task Planning Based on Visual Language Models and Zero-sum Games

From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models

ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling

RelationVLM: Making Large Vision-Language Models Understand Visual Relations

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models

Test-time Distribution Learning Adapter for Cross-modal Visual Reasoning

RegionGPT: Towards Region Understanding Vision Language Model

GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Events

Large Language Models as Automated Aligners for benchmarking Vision-Language Models

Systematic Visual Reasoning through Object-Centric Relational Abstraction

Benchmarking Sequential Visual Input Reasoning and Prediction in Multimodal Large Language Models