视觉大模型 / VLM - arXivDaily 专题

2505.08910 2025-05-16 cs.CV cs.CL 85%

Behind Maya: Building a Multilingual Vision Language Model

Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, S M Iftekhar Uddin, Shayekh Bin Islam, Roshan Santhosh, Snegha A, Drishti Sharma, Chen Liu, Isha Chaturvedi, Genta Indra Winata, Ashvanth. S, Snehanshu Mukherjee, Alham Fikri Aji

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted at VLMs4ALL CVPR 2025 Workshop; corrected workshop name spelling

2505.03374 2025-05-07 cs.CV 85%

Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models

Abram Schonfeldt, Benjamin Maylor, Xiaofang Chen, Ronald Clark, Aiden Doherty

机构 * Department of Population Health, University of Oxford（牛津大学流行病学与健康统计学系）； University of Oxford（牛津大学）； School of Epidemiology and Health Statistics, Chengdu Medical College（成都医学院流行病学与健康统计学系）

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

2501.18624 2025-02-10 cs.CR cs.AI 85%

Membership Inference Attacks Against Vision-Language Models

Yuke Hu, Zheng Li, Zhihao Liu, Yang Zhang, Zhan Qin, Kui Ren, Chun Chen

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.AI

Comments Accepted by USENIX'25; 22 pages, 28 figures;

2501.04322 2025-01-24 cs.CV 85%

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts

Miao Rang, Zhenni Bi, Chuanjian Liu, Yehui Tang, Kai Han, Yunhe Wang

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

2501.12327 2025-01-22 cs.CV 85%

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

Xianwei Zhuang, Yuxin Xie, Yufan Deng, Liming Liang, Jinghan Ru, Yuguo Yin, Yuexian Zou

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2412.12359 2025-01-08 cs.CV cs.CL 85%

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering

Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, Yunpu Ma

专题命中 VLM训练与架构：LLaVA(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2412.20105 2024-12-31 cs.CV 85%

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to AAAI2025

2311.12668 2024-12-10 cs.AI cs.CE 85%

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Cyril Picard, Kristen M. Edwards, Anna C. Doris, Brandon Man, Giorgio Giannone, Md Ferdous Alam, Faez Ahmed

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；LLaVA(abstract)；分类 cs.AI

2412.00556 2024-12-10 cs.CV 85%

Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction

Shiyu Zhao, Zhenting Wang, Felix Juefei-Xu, Xide Xia, Miao Liu, Xiaofang Wang, Mingfu Liang, Ning Zhang, Dimitris N. Metaxas, Licheng Yu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments Technical report, 18 pages

2412.00832 2024-12-03 cs.CV 85%

EventGPT: Event Stream Understanding with Multimodal Large Language Models

Shaoyu Liu, Jianing Li, Guanghui Zhao, Yunjian Zhang, Xin Meng, Fei Richard Yu, Xiangyang Ji, Ming Li

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2411.14505 2024-11-25 cs.CV 85%

LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval

Weiheng Lu, Jian Li, An Yu, Ming-Ching Chang, Shengpeng Ji, Min Xia

专题命中 VLM训练与架构：LLaVA(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2410.13859 2024-10-18 cs.CV 85%

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, Rongrong Ji

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2409.02278 2024-09-05 cs.CV 85%

Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems

Sanjita Prajapati, Tanu Singh, Chinmay Hegde, Pranamesh Chakraborty

专题命中 VLM训练与架构：visual language model(title)；vision language model(abstract)；VLM(abstract)；LLaVA(abstract)

2406.09913 2024-08-27 cs.CV 85%

OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design

Zhe Yuan, Jianqi Shi, Yanhong Huang

专题命中 VLM训练与架构：visual language model(title,abstract)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Journal ref Computers & Graphics 124C (2024) 104048

2408.06721 2024-08-14 cs.CV 85%

Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities

Shivam Chandhok, Wan-Cyuan Fan, Leonid Sigal

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Under Submission

2408.03940 2024-08-08 cs.CV 85%

How Well Can Vision Language Models See Image Details?

Chenhui Gou, Abdulwahab Felemban, Faizan Farooq Khan, Deyao Zhu, Jianfei Cai, Hamid Rezatofighi, Mohamed Elhoseiny

专题命中 VLM训练与架构：vision language model(title,abstract)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2406.10165 2024-06-17 cs.CV cs.RO 85%

CarLLaVA: Vision language models for camera-only closed-loop driving

Katrin Renz, Long Chen, Ana-Maria Marcu, Jan Hünermann, Benoit Hanotte, Alice Karnsund, Jamie Shotton, Elahe Arani, Oleg Sinavski

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

Comments Outstanding Champion & Innovation Award @ CARLA Autonomous Driving Challenge 2024; Project video: https://youtu.be/E1nsEgcHRuc

2312.07533 2024-05-20 cs.CV 85%

VILA: On Pre-training for Visual Language Models

Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, Song Han

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

Comments CVPR 2024

2403.03003 2024-03-06 cs.CV 85%

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, Rongrong Ji

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2304.00685 2024-02-19 cs.CV 85%

Vision-Language Models for Vision Tasks: A Survey

Jingyi Zhang, Jiaxing Huang, Sheng Jin, Shijian Lu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；visual language model(abstract)；分类 cs.CV

2401.16160 2024-01-31 cs.CV 85%

LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs

Shaoxiang Chen, Zequn Jie, Lin Ma

专题命中 VLM训练与架构：LLaVA(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2410.20971 2025-02-13 cs.CV cs.AI cs.LG 85%

BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks

Yunhan Zhao, Xiang Zheng, Lin Luo, Yige Li, Xingjun Ma, Yu-Gang Jiang

专题命中 VLM训练与架构：vision-language model(title,journal_ref)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref ICLR 2025, BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks. In Proceedings of the International Conference on Learning Representations (ICLR), 2025

2311.05437 2023-11-10 cs.CV cs.AI cs.CL cs.LG cs.MM 85%

LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents

Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang, Jianfeng Gao, Chunyuan Li

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 25 pages, 25M file size. Project Page: https://llava-vl.github.io/llava-plus/

2307.03135 2023-10-13 cs.CV cs.AI cs.CL cs.LG 85%

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability

Xuanlin Li, Yunhao Fang, Minghua Liu, Zhan Ling, Zhuowen Tu, Hao Su

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract,comments)；分类 cs.CV、cs.AI、cs.LG

Comments Published at International Conference on Computer Vision (ICCV) 2023. Poster at https://xuanlinli17.github.io/pdfs/iccv23_large_vlm_distillation_poster.pdf

2304.01457 2023-06-22 cs.AI cs.CV cs.LG 85%

Exploring Vision-Language Models for Imbalanced Learning

Yidong Wang, Zhuohao Yu, Jindong Wang, Qiang Heng, Hao Chen, Wei Ye, Rui Xie, Xing Xie, Shikun Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract,comments)；分类 cs.CV、cs.AI、cs.LG

Comments IJCV minor revision; 16 pages; code: https://github.com/Imbalance-VLM/Imbalance-VLM

2608.07193 2026-08-10 cs.LG cs.CV 新提交 85%

An AI4AI Framework for Visual Token Pruning

用于视觉令牌剪枝的AI4AI框架

Zhen Liu, Wenli Huang, Wei Song, Yuhan Liu, Zhiqin Yang, Jingwen Fu

专题命中 VLM训练与架构：MLLM(summary_cn,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

AI总结本研究提出AutoPrune框架，通过TPDSL让LLM自动设计视觉令牌剪枝策略，在14个多模态基准和3个MLLM主干上，剪枝94.4%视觉令牌仍保留超99%性能，大幅降低推理成本。

2604.27975 2026-08-05 cs.CV cs.AI 版本更新 85%

TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions

TransVLM：一种用于检测任意 shot 转换的视觉-语言框架和基准

Ce Chen, Yi Ren, Yuanming Li, Viktor Goriachko, Zhenhui Ye, Zujin Guo, Zhibin Hong, Mingming Gong

机构 * University of Melbourne（墨尔本大学）； HeyGen Research（HeyGen研究院）； Nanyang Technological University（南洋理工大学）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 TransVLM 提出了一种视觉-语言框架，通过注入光流作为关键运动先验，提升对任意 shot 转换的检测能力，同时设计了可扩展的数据引擎和基准测试，实验表明其性能优于传统方法和现有 VLM。

Comments v2: Corrected an inaccurate statement in the abstract regarding production deployment. Accepted by ECCV 2026. For more related research, please visit HeyGen Research (https://www.heygen.com/research) and HeyGen Avatar-V (https://www.heygen.com/research/avatar-v-model). Project page: https://chence17.github.io/TransVLM/

2603.11804 2026-08-04 cs.CV cs.LG 版本更新 85%

OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs

基于OpenStreetMap的领域自适应方法：为遥感VLMs的领域适应

Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi, Delyan Boychev, Luc Van Gool, Danda Pani Paudel

机构 * INSAIT, Sofia University ``St. Kliment Ohridski''（INSAIT，索菲亚大学『圣克莱门特·奥赫里德斯』）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.LG

AI总结本文提出OSMDA方法，利用OpenStreetMap数据生成自标注数据，无需人工标注或强外部模型，通过细调基础VLM实现遥感领域适应，经10个基准测试显示其在文本生成任务中达到SOTA，且训练成本更低。

2607.27699 2026-07-31 cs.CV cs.AI 新提交 85%

RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation

RefineSVG：视觉反馈驱动的图像转SVG生成强化学习

Shaobo Liu, Feiqiao Mao, Shuaishuai Zhou, Yan Zhan, Weiqi Tan, Zhiqiong Lu, Zhengping Liang

机构 * Shenzhen University（深圳大学）； Peking University（北京大学）

专题命中 VLM训练与架构：MLLM(summary_cn,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 RefineSVG是一种视觉反馈驱动的闭环框架，通过引入SVG语义词汇表和渐进式训练，使MLLM实现高保真图像转SVG生成，性能优于现有基线。

Comments 17 pages, 5 main-paper figures. Accepted at the 34th ACM International Conference on Multimedia (ACM MM 2026). Includes the complete supplementary material

2607.23368 2026-07-28 cs.CV cs.AI cs.CL 新提交 85%

Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing

用基于树图解析支持的加权班扎夫交互解释生物医学CLIP

Jakub Rymarski, Adam Rempała, Bartłomiej Sobieski, Przemysław Biecek

机构 * University of Warsaw（华沙大学）； Centre for Credible AI, Warsaw University of Technology（华沙理工大学可信人工智能中心）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结研究针对视觉语言模型在医学任务中解释难的问题，引入ParseFIxLIP方法，将树图解析融入班扎夫交互博弈，通过smart_depth分组策略减轻概念碎片化，提升跨模态交互可解释性，为VLM决策提供医学领域相关见解。

Comments 12 pages, 13 figures. Accepted at EXPLIMED 2026 (Third Workshop on Explainable Artificial Intelligence for the medical domain), IJCAI-ECAI 2026

1. VLM训练与架构 5009 篇

Behind Maya: Building a Multilingual Vision Language Model

Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models

Membership Inference Attacks Against Vision-Language Models

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction

EventGPT: Event Stream Understanding with Multimodal Large Language Models

LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems

OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design

Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities

How Well Can Vision Language Models See Image Details?

CarLLaVA: Vision language models for camera-only closed-loop driving

VILA: On Pre-training for Visual Language Models

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

Vision-Language Models for Vision Tasks: A Survey

LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs

BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks

LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability

Exploring Vision-Language Models for Imbalanced Learning

An AI4AI Framework for Visual Token Pruning

TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions

OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs

RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation

Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing