视觉大模型 / VLM - arXivDaily 专题

2512.24555 2026-01-21 cs.LG 74%

From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme

从感知到 punchline：通过野生表情包艺术赋能 VLM

Xueyan Li, Yingyi Xue, Mengjie Jiang, Qingzi Zhu, Yazhe Niu

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）； School of Software Engineering（软件工程学院）； Columbia Engineering（哥伦比亚工程学院）； Columbia University（哥伦比亚大学）； The Chinese University of Hong Kong MMLab（香港中文大学MMLab）

专题命中 VLM训练与架构：VLM(title)；分类 cs.LG

AI总结 HUMOR 通过分层推理和群体偏好对齐，提升 VLM 在多模态生成中的推理多样性与幽默质量。

Comments 46 pages, 20 figures

2506.12409 2026-01-12 cs.CV 74%

Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Models

分支还是层？零阶优化用于视觉-语言模型的持续学习

Ziwei Liu, Borui Kang, Wei Li, Hangjie Yuan, Yanbing Yang, Wenbin Li, Yifan Zhu, Tao Feng, Jun Luo

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

AI总结本文提出一种基于零阶优化的视觉-语言模型持续学习方法，通过模态感知策略提升模型逃避局部极小值的能力，实验表明在四个基准测试中取得最佳性能。

Comments Published in the Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2026)

2512.10262 2025-12-12 cs.CV 74%

VLM-NCD:Novel Class Discovery with Vision-Based Large Language Models

基于视觉大语言模型的新型类别发现

Yuetong Su, Baoguo Wei, Xinyu Wang, Xu Li, Lixin Li

专题命中 VLM训练与架构：VLM(title)；分类 cs.CV

AI总结基于视觉大语言模型的新型类别发现方法，通过融合视觉-文本语义和原型引导聚类，提升未知类别分类准确率25.3%，并展现对长尾分布的鲁棒性。

Comments 8 pages, 5 figures, conference

2508.02671 2025-11-13 cs.CV 74%

Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models

Haoyang Li, Liang Wang, Chao Wang, Siyu Zhou, Jing Jiang, Yan Peng, Guodong Long

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments 16 pages, 6 figures, 15 tables

2504.03724 2025-11-04 cs.CV cs.CL 74%

CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward

Zhiqiang Wang, Pengbin Feng, Yanbin Lin, Shuzhang Cai, Zongao Bian, Jinghua Yan, Xingquan Zhu

机构 * Florida Atlantic University（佛罗里达大学）； University of Southern California（南加州大学）； University of Texas at Dallas（德克萨斯大学达拉斯分校）； Georgia Institute of Technology（佐治亚理工学院）； University of Utah（犹他大学）

专题命中 VLM训练与架构：vision language model(title)；分类 cs.CV

Comments 10 pages, 6 figures and 4 tables

Journal ref 2025 IEEE International Conference on Big Data (IEEE BigData 2025)

2510.21606 2025-10-27 cs.CV 74%

Modest-Align: Data-Efficient Alignment for Vision-Language Models

Jiaxiang Liu, Yuan Wang, Jiawei Du, Joey Tianyi Zhou, Mingkun Xu, Zuozhu Liu

机构 * Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）； ZJU-Angelalign R&D Center for Intelligence Healthcare（浙大天使align智能医疗研发中心）； Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）； Agency for Science, Technology and Research (A*STAR)（科技研究局）； Institute of High Performance Computing (IHPC)（高性能计算研究所）

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

2409.05381 2025-09-23 cs.CV 74%

Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models

Xudong Li, Zihao Huang, Yan Zhang, Yunhang Shen, Ke Li, Xiawu Zheng, Liujuan Cao, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）； Beijing Institute of Technology（北京理工大学）； Tencent Youtu Lab, Shanghai, China（腾讯优图实验室，中国）

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

2408.01437 2025-09-22 cs.CV cs.GR 74%

Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorization

Yang You, Mikaela Angelina Uy, Jiaqi Han, Rahul Thomas, Haotong Zhang, Yi Du, Hansheng Chen, Francis Engelmann, Suya You, Leonidas Guibas

机构 * Stanford University（斯坦福大学）； NVIDIA（英伟达）； Peking University（北京大学）； DEVCOM Army Research Laboratory（陆军研究实验室）

专题命中 VLM训练与架构：VLM(title)；分类 cs.CV

Comments Accepted to SIGGRAPH Asia 2025

2509.00039 2025-09-03 cs.CV 74%

AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models

Yuqi Li, Chuanguang Yang, Junhao Dong, Zhengtao Yao, Haoyan Xu, Zeyu Dong, Hansheng Zeng, Zhulin An, Yingli Tian

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments 9 pages

2301.12429 2025-08-14 cs.CV 74%

Debiased Fine-Tuning for Vision-language Models by Prompt Regularization

Beier Zhu, Yulei Niu, Saeil Lee, Minhoe Hur, Hanwang Zhang

机构 * NTU（国立新加坡大学）

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments AAAI2023 accepted

2311.17963 2025-08-08 cs.CV 74%

M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation

Xiaowei Chi, Junbo Qi, Rongyu Zhang, Shanghang Zhang, Qifeng Liu, Yike Guo

机构 * The Hong Kong University of Science and Technology（香港科学与技术大学）； Waseda University（早稻田大学）； Peking University（北京大学）

专题命中 VLM训练与架构：VLM(title)；分类 cs.CV

2507.04107 2025-07-23 cs.CV 74%

VICI: VLM-Instructed Cross-view Image-localisation

Xiaohan Zhang, Tavis Shore, Chen Chen, Oscar Mendez, Simon Hadfield, Safwan Wshah

机构 * University of Vermont（佛罗里达大学）； University of Surrey（塞弗尔大学）； University of Central Florida（佛罗里达中央大学）； Locus Robotics（Locus机器人公司）

专题命中 VLM训练与架构：VLM(title)；分类 cs.CV

2402.09816 2025-07-21 cs.CV 74%

Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment

Angelos Zavras, Dimitrios Michail, Begüm Demir, Ioannis Papoutsis

机构 * organization= Orion Lab, National Observatory of Athens \& National Technical University of Athens , country= Greece ； organization= Department of Informatics \& Telematics, Harokopio University of Athens , country= Greece ； organization= Faculty of Electrical Engineering ； organization= BIFOLD - Berlin Institute for the Foundations of Learning

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments Accepted at the ISPRS Journal of Photogrammetry and Remote Sensing. Our code implementation and weights for all experiments are publicly available at https://github.com/Orion-AI-Lab/MindTheModalityGap

2506.21863 2025-06-30 cs.CV 74%

Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling

Sungjune Park, Yeongyun Kim, Se Yeon Kim, Yong Man Ro

机构 * Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（整合视觉与语言实验室，电气工程学院，韩国科学技术院（KAIST））

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments 13 pages including reference pages, 7 tables, and 6 figures

2506.06406 2025-06-26 cs.CL cs.AI 74%

SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities

Guoyang Xia, Yifeng Ding, Fengfa Li, Lei Ren, Wei Chen, Fangxiang Feng, Xiaojie Wang

专题命中 VLM训练与架构：multimodal large language model(title)；分类 cs.AI

2504.00839 2025-06-24 cs.RO cs.AI 74%

Context-Aware Human Behavior Prediction Using Multimodal Large Language Models: Challenges and Insights

Yuchen Liu, Lino Lerch, Luigi Palmieri, Andrey Rudenko, Sebastian Koch, Timo Ropinski, Marco Aiello

机构 * Corporate Sector Research and Advance Engineering, Robert Bosch GmbH（罗伯特博世集团企业部门研究与先进工程）； Service Computing Department, Institute of Architecture of Application Systems, University of Stuttgart（斯图加特大学应用系统研究所服务计算部门）； Visual Computing Group, Ulm University（乌尔姆大学视觉计算组）

专题命中 VLM训练与架构：multimodal large language model(title)；分类 cs.AI

Comments Accepted at IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2025

2506.13964 2025-06-18 eess.IV cs.LG 74%

Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography

Yusdivia Molina-Román, David Gómez-Ortiz, Ernestina Menasalvas-Ruiz, José Gerardo Tamez-Peña, Alejandro Santos-Díaz

机构 * School of Engineering and Sciences（工程与科学学院）； School of Medical and Health Sciences（医学与健康科学学院）

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.LG

Comments 6 pages, 4 figures

2405.15289 2025-06-18 cs.CV 74%

Learning Invariant Causal Mechanism from Vision-Language Models

Zeen Song, Siyu Zhao, Xingyu Zhang, Jiangmeng Li, Changwen Zheng, Wenwen Qiang

机构 * Institute of Software Chinese Academy of Sciences, Beijing, China（中国科学院软件研究所）； University of the Chinese Academy of Sciences（中国科学院大学）

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments Accepted to ICML 2025

2503.13443 2025-03-25 cs.CV cs.MM 74%

DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models

Haoyang Li, Liang Wang, Chao Wang, Jing Jiang, Yan Peng, Guodong Long

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments Accepted by the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025 (CVPR 2025)

2502.03230 2025-02-06 cs.CV cs.MM 74%

Efficient Vision Language Model Fine-tuning for Text-based Person Anomaly Search

Jiayi He, Shengeng Tang, Ao Liu, Lechao Cheng, Jingjing Wu, Yanyan Wei

专题命中 VLM训练与架构：vision language model(title)；分类 cs.CV

Comments Accepted by 2025 WWW Workshop on MORE

2408.09916 2025-01-24 cs.CV cs.CL 74%

Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit

Qizhou Chen, Taolin Zhang, Chengyu Wang, Xiaofeng He, Dakan Wang, Tingting Liu

专题命中 VLM训练与架构：vision language model(title)；分类 cs.CV

Comments Accepted to AAAI-2025 as an oral presentation

2412.15277 2024-12-23 cs.CL cs.AI 74%

PLPP: Prompt Learning with Perplexity Is Self-Distillation for Vision-Language Models

Biao Liu, Wenyi Fang, Xiaoyu Wu, Yang Zheng, Zheng Hu, Bo Yuan

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.AI

2408.10188 2024-12-16 cs.CV cs.CL 74%

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

专题命中 VLM训练与架构：visual language model(title)；分类 cs.CV

Comments Code and models are available at https://github.com/NVlabs/VILA/tree/main/longvila

2409.14485 2024-12-11 cs.CV 74%

Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding

Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, Bo Zhao

专题命中 VLM训练与架构：vision language model(title)；分类 cs.CV

2411.16018 2024-11-26 cs.CV 74%

Style-Pro: Style-Guided Prompt Learning for Generalizable Vision-Language Models

Niloufar Alipour Talemi, Hossein Kashiani, Fatemeh Afghah

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments Accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2025)

2410.04223 2024-10-08 cs.LG physics.chem-ph q-bio.BM 74%

Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning

Gang Liu, Michael Sun, Wojciech Matusik, Meng Jiang, Jie Chen

专题命中 VLM训练与架构：multimodal large language model(title)；分类 cs.LG

Comments 27 pages, 11 figures, 4 tables

2409.04693 2024-09-10 cs.AI 74%

MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality

Ruiting Dai, Yuqiao Tan, Lisi Mo, Tao He, Ke Qin, Shuang Liang

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.AI

2310.03103 2024-08-30 cs.LG 74%

Learning to Prompt Your Domain for Vision-Language Models

Guoyizhe Wei, Feng Wang, Anshul Shah, Rama Chellappa

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.LG

2408.14812 2024-08-28 cs.CV 74%

HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling

Yubin Wang, Xinyang Jiang, De Cheng, Wenli Sun, Dongsheng Li, Cairong Zhao

专题命中 VLM训练与架构：vision-language model(title)；分类 cs.CV

Comments 19 pages, 7 figures, 7 tables. arXiv admin note: substantial text overlap with arXiv:2312.06323

2408.12867 2024-08-26 cs.CV 74%

Semantic Alignment for Multimodal Large Language Models

Tao Wu, Mengze Li, Jingyuan Chen, Wei Ji, Wang Lin, Jinyang Gao, Kun Kuang, Zhou Zhao, Fei Wu

专题命中 VLM训练与架构：multimodal large language model(title)；分类 cs.CV

Comments Accepted by MM 2024

1. VLM训练与架构 5048 篇

From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme

Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Models

VLM-NCD:Novel Class Discovery with Vision-Based Large Language Models

Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models

CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward

Modest-Align: Data-Efficient Alignment for Vision-Language Models

Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models

Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorization

AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models

Debiased Fine-Tuning for Vision-language Models by Prompt Regularization

M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation

VICI: VLM-Instructed Cross-view Image-localisation

Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment

Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling

SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities

Context-Aware Human Behavior Prediction Using Multimodal Large Language Models: Challenges and Insights

Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography

Learning Invariant Causal Mechanism from Vision-Language Models

DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models

Efficient Vision Language Model Fine-tuning for Text-based Person Anomaly Search

Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit

PLPP: Prompt Learning with Perplexity Is Self-Distillation for Vision-Language Models

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding

Style-Pro: Style-Guided Prompt Learning for Generalizable Vision-Language Models

Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning

MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality

Learning to Prompt Your Domain for Vision-Language Models

HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling

Semantic Alignment for Multimodal Large Language Models