视觉大模型 / VLM - arXivDaily 专题

2502.01828 2025-05-05 cs.RO cs.LG 83%

From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment

Yilin Wu, Ran Tian, Gokul Swamy, Andrea Bajcsy

机构 * Carnegie Mellon University（卡内基梅隆大学）； UC Berkeley（伯克利大学）

专题命中 VLM训练与架构：VLM(title,abstract)；vision language model(abstract)；分类 cs.LG

2505.00134 2025-05-02 cs.CV 83%

Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design

Vasudev Sharma, Ahmed Alagha, Abdelhakim Khellaf, Vincent Quoc-Huy Trinh, Mahdi S. Hosseini

机构 * Department of Computer Science and Software Engineering (CSSE) Concordia University（计算机科学与软件工程系（CSSE）康科迪亚大学）； University of Montreal Hospital Center（蒙特利尔大学医院中心）； Mila–Quebec AI Institute（魁北克人工智能研究所）

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2503.10742 2025-04-25 cs.LG cs.CL 83%

Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing

Yudong Liu, Jingwei Sun, Yueqian Lin, Jingyang Zhang, Ming Yin, Qinsi Wang, Jianyi Zhang, Hai Li, Yiran Chen

机构 * Duke University（杜克大学）； Duke Kunshan University（杜克昆山大学）

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.LG

2504.05457 2025-04-09 cs.CV 83%

Taxonomy-Aware Evaluation of Vision-Language Models

Vésteinn Snæbjarnarson, Kevin Du, Niklas Stoehr, Serge Belongie, Ryan Cotterell, Nico Lang, Stella Frank

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments CVPR 2025

2501.17131 2025-04-08 cs.CV 83%

Scenario Understanding of Traffic Scenes Through Large Visual Language Models

Esteban Rivera, Jannik Lübberstedt, Nico Uhlemann, Markus Lienkamp

专题命中 VLM训练与架构：visual language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted at WACV2025

2504.01934 2025-04-04 cs.CV 83%

ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement

Runhui Huang, Chunwei Wang, Junwei Yang, Guansong Lu, Yunlong Yuan, Jianhua Han, Lu Hou, Wei Zhang, Lanqing Hong, Hengshuang Zhao, Hang Xu

专题命中 VLM训练与架构：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

2503.20502 2025-04-01 cs.CV 83%

MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning

Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Jiayi Ji, Jie Lou, Debing Zhang, Rongrong Ji

专题命中 VLM训练与架构：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Tech Report

2503.19839 2025-04-01 cs.CV 83%

FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model

Jun Zhou, Jiahao Li, Zunnan Xu, Hanhui Li, Yiji Cheng, Fa-Ting Hong, Qin Lin, Qinglin Lu, Xiaodan Liang

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

2503.17238 2025-03-24 cs.CV 83%

Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology

Devavrat Tomar, Guillaume Vray, Dwarikanath Mahapatra, Sudipta Roy, Jean-Philippe Thiran, Behzad Bozorgtabar

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to ISBI 2025

2409.11905 2025-03-24 cs.RO cs.AI cs.IR 83%

AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robots

Zhaxizhuoma Zhaxizhuoma, Pengan Chen, Ziniu Wu, Jiawei Sun, Dong Wang, Peng Zhou, Nieqing Cao, Yan Ding, Bin Zhao, Xuelong Li

专题命中 VLM训练与架构：VLM(title,abstract)；LLaVA(abstract)；分类 cs.AI

2503.14478 2025-03-20 cs.CV 83%

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, Dahua Lin

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Evaluation Code and dataset see https://github.com/open-compass/Creation-MMBench

2406.13642 2025-03-20 cs.CV 83%

SpatialBot: Precise Spatial Understanding with Vision Language Models

Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, Bo Zhao

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2503.13171 2025-03-18 cs.RO cs.AI 83%

HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learning

Wensheng Wang, Ning Tan

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.AI

2503.12731 2025-03-18 cs.CV 83%

Navigating Heat Exposure: Simulation of Route Planning Based on Visual Language Model Agents

Haoran Ma, Kaihan Zhang, Jiannan Cai

专题命中 VLM训练与架构：visual language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 10 pages, 6 figures

2503.11549 2025-03-17 cs.CV 83%

Similarity-Aware Token Pruning: Your VLM but Faster

Ahmadreza Jeddi, Negin Baghbanzadeh, Elham Dolatabadi, Babak Taati

专题命中 VLM训练与架构：VLM(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments 15 pages, 8 figures, 8 tables

2503.11368 2025-03-17 cs.CV 83%

PBR3DGen: A VLM-guided Mesh Generation with High-quality PBR Texture

Xiaokang Wei, Bowen Zhang, Xianghui Yang, Yuxuan Wang, Chunchao Guo, Xi Zhao, Yan Luximon

专题命中 VLM训练与架构：VLM(title,abstract)；vision language model(abstract)；分类 cs.CV

Comments Homepage: https://pbr3dgen1218.github.io/

2503.10501 2025-03-14 cs.CV 83%

TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models

Xudong Tan, Peng Ye, Chongjun Tu, Jianjian Cao, Yaoxin Yang, Lin Zhang, Dongzhan Zhou, Tao Chen

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2501.01428 2025-03-12 cs.CV 83%

GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, Hengshuang Zhao

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Project page: https://gpt4scene.github.io/

2503.07075 2025-03-11 cs.CV 83%

XR-VLM: Cross-Relationship Modeling with Multi-part Prompts and Visual Features for Fine-Grained Recognition

Chuanming Wang, Henming Mao, Huanhuan Zhang, Huiyuan Fu, Huadong Ma

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2411.12915 2025-03-05 cs.CV 83%

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Vishwesh Nath, Wenqi Li, Dong Yang, Andriy Myronenko, Mingxin Zheng, Yao Lu, Zhijian Liu, Hongxu Yin, Yucheng Tang, Pengfei Guo, Can Zhao, Ziyue Xu, Yufan He, Greg Heinrich, Yee Man Law, Benjamin Simon, Stephanie Harmon, Stephen Aylward, Marc Edgar, Michael Zephyr, Song Han, Pavlo Molchanov, Baris Turkbey, Holger Roth, Daguang Xu

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

2410.11087 2025-03-05 cs.CV 83%

Locality Alignment Improves Vision-Language Models

Ian Covert, Tony Sun, James Zou, Tatsunori Hashimoto

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments ICLR 2025 Camera-Ready

2502.20108 2025-03-04 cs.CV cs.RO 83%

VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers

Ziang Guo, Konstantin Gubernatorov, Selamawit Asfaw, Zakhar Yagudin, Dzmitry Tsetserukou

专题命中 VLM训练与架构：VLM(title,abstract)；visual language model(abstract)；分类 cs.CV

Comments Submitted paper

2501.09695 2025-03-04 cs.CV 83%

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, Dongsheng Li

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

2409.18459 2025-03-04 cs.CV cs.MM 83%

FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation

Yuki Imajuku, Yoko Yamakata, Kiyoharu Aizawa

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 15 pages, 5 figures. We found errors in the calculation of evaluation metrics, which were corrected in this version with $\color{blue}{\text{modifications highlighted in blue}}$. Please also see the Appendix

2410.05340 2025-03-03 cs.LG 83%

Generating CAD Code with Vision-Language Models for 3D Designs

Kamel Alrashedy, Pradyumna Tambwekar, Zulfiqar Zaidi, Megan Langwasser, Wei Xu, Matthew Gombolay

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

2410.17247 2025-02-28 cs.CV cs.CL 83%

PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, Dahua Lin

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments CVPR 2025, code is available at https://github.com/Cooperx521/PyramidDrop

2412.13333 2025-02-26 cs.LG 83%

Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality

Qitong Wang, Tang Li, Kien X. Nguyen, Xi Peng

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

Comments In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), 2025

2502.10250 2025-02-25 cs.CL cs.CV 83%

VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

Gokul Karthik Kumar, Iheb Chaabane, Kebin Wu

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted at PAKDD 2025

2501.18623 2025-02-19 cs.CV cs.GR 83%

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments ICLR 2025 Spotlight

2410.04780 2025-02-19 cs.CV 83%

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, Xuming Hu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by The Thirteenth International Conference on Learning Representations (ICLR 2025)

1. VLM训练与架构 5030 篇

From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment

Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design

Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing

Taxonomy-Aware Evaluation of Vision-Language Models

Scenario Understanding of Traffic Scenes Through Large Visual Language Models

ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement

MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning

FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model

Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology

AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robots

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

SpatialBot: Precise Spatial Understanding with Vision Language Models

HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learning

Navigating Heat Exposure: Simulation of Route Planning Based on Visual Language Model Agents

Similarity-Aware Token Pruning: Your VLM but Faster

PBR3DGen: A VLM-guided Mesh Generation with High-quality PBR Texture

TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models

GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

XR-VLM: Cross-Relationship Modeling with Multi-part Prompts and Visual Features for Fine-Grained Recognition

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Locality Alignment Improves Vision-Language Models

VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation

Generating CAD Code with Vision-Language Models for 3D Designs

PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality

VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality