视觉大模型 / VLM - arXivDaily 专题

2501.00574 2025-07-15 cs.CV cs.LG 73%

VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, Yu Qiao, Yali Wang, Limin Wang

机构 * Shanghai AI Laboratory（上海人工智能实验室）； Nanjing University（南京大学）； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）； Shanghai Innovation Institute（上海创新研究院）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

2507.08610 2025-07-14 cs.LG cs.CV 73%

Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data

Parag Dutta, Ambedkar Dukkipati

机构 * Department of Computer Science and Automation（计算机科学与自动化系）； Indian Institute of Science（印度科学院）

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2507.05513 2025-07-09 cs.CV cs.AI 73%

Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model

Mengyao Xu, Gabriel Moreira, Ronay Ak, Radek Osmulski, Yauhen Babakhin, Zhiding Yu, Benedikt Schifferer, Even Oldridge

机构 * Proceedings of Conference XXX（会议论文集）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2506.10967 2025-07-02 cs.CV cs.AI 73%

Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs

Qizhe Zhang, Mengzhen Liu, Lichen Li, Ming Lu, Yuan Zhang, Junwen Pan, Qi She, Shanghang Zhang

机构 * National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（国家多媒体信息处理重点实验室，计算机学院，北京大学）； ByteDance（字节跳动）

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 22 pages, 5 figures, code: https://github.com/Theia-4869/CDPruner, project page: https://theia-4869.github.io/CDPruner

2412.03704 2025-07-02 cs.CV cs.CL cs.LG 73%

Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehension

Xiyao Wang, Zhengyuan Yang, Linjie Li, Hongjin Lu, Yuancheng Xu, Chung-Ching Lin, Kevin Lin, Furong Huang, Lijuan Wang

机构 * University of Maryland, College Park（马里兰大学学院公园分校）； Microsoft（微软公司）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2411.12787 2025-07-02 cs.CV cs.AI 73%

From Holistic to Localized: Local Enhanced Adapters for Efficient Visual Instruction Fine-Tuning

Pengkun Jiao, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang

机构 * Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University（上海智能信息处理关键实验室，复旦大学计算机学院）； Shanghai Collaborative Innovation Center on Intelligent Visual Computing（上海智能视觉计算协同创新中心）； Singapore Management University（新加坡管理大学）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2025

2506.23115 2025-07-01 cs.CV cs.AI cs.CL 73%

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings

Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, Zhicheng Dou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）； Stanford University（斯坦福大学）； Microsoft Corporation（微软公司）

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Homepage: https://haon-chen.github.io/MoCa/

2506.21017 2025-06-27 cs.CV cs.AI 73%

Multimodal Prompt Alignment for Facial Expression Recognition

Fuyan Ma, Yiran He, Bin Sun, Shutao Li

机构 * Chinese Academy of Military Science（中国军事科学院）； Changchun University of Science and Technology（长春理工大学）； Hunan University（湖南大学）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments To appear in ICCV2025

2506.09691 2025-06-12 cs.CV cs.CL cs.LG 73%

Adding simple structure at inference improves Vision-Language Compositionality

Imanol Miranda, Ander Salaberria, Eneko Agirre, Gorka Azkune

机构 * HiTZ Center – Ixa, University of the Basque Country (UPV/EHU)（希茨中心–Ixa，巴斯克大学（UPV/EHU））

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2505.16933 2025-06-05 cs.LG cs.CL cs.CV 73%

LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning

Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, Chongxuan Li

机构 * Gaoling School of AI, Renmin University of China（中国人民大学人工智能学院）； Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大型模型与智能治理研究重点实验室）； Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（下一代智能搜索与推荐工程技术研究中心，教育部）； Ant Group（蚂蚁集团）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

Comments Project page and codes: \url{https://ml-gsai.github.io/LLaDA-V-demo/}

2506.02020 2025-06-04 cs.CV cs.LG 73%

Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifying

Youze Xue, Dian Li, Gang Liu

机构 * Tencent（腾讯）

专题命中 VLM训练与架构：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

2506.01955 2025-06-03 cs.CV cs.CL cs.LG 73%

Dual-Process Image Generation

Grace Luo, Jonathan Granskog, Aleksander Holynski, Trevor Darrell

机构 * UC Berkeley（伯克利大学）； Runway

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2505.24541 2025-06-02 cs.CV cs.AI 73%

Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts

Xin He, Xumeng Han, Longhui Wei, Lingxi Xie, Qi Tian

机构 * Huawei Inc.（华为公司）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2408.05211 2025-06-02 cs.CV cs.AI cs.CL 73%

VITA: Towards Open-Source Interactive Omni Multimodal LLM

Chaoyou Fu, Haojia Lin, Zuwei Long, Yunhang Shen, Yuhang Dai, Meng Zhao, Yi-Fan Zhang, Shaoqi Dong, Yangze Li, Xiong Wang, Haoyu Cao, Di Yin, Long Ma, Xiawu Zheng, Rongrong Ji, Yunsheng Wu, Ran He, Caifeng Shan, Xing Sun

机构 * NJU（南京大学）； Tencent Youtu Lab（腾讯计算机系统有限公司视觉实验室）； XMU（厦门大学）； CASIA（中国科学院自动化研究所）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://vita-home.github.io

2505.21647 2025-05-29 cs.CV cs.LG 73%

QuARI: Query Adaptive Retrieval Improvement

Eric Xing, Abby Stylianou, Robert Pless, Nathan Jacobs

机构 * Washington University in St. Louis（华盛顿大学圣路易斯分校）； Saint Louis University（圣路易斯大学）； The George Washington University（乔治华盛顿大学）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments 13 pages, 4 figures, 4 tables

2409.19291 2025-05-29 cs.CV cs.AI 73%

CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling

Jihai Zhang, Xiaoye Qu, Tong Zhu, Yu Cheng

机构 * First Author Affiliation（第一作者机构）； Second Author Affiliation（第二作者机构）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2505.20053 2025-05-27 cs.CV cs.AI cs.CL cs.MM 73%

Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion

Zheqi Lv, Junhao Chen, Qi Tian, Keting Yin, Shengyu Zhang, Fei Wu

机构 * Zhejiang University（浙江大学）

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2504.16083 2025-05-26 cs.CV cs.LG 73%

MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention

Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments Accepted at ICML 2025

2505.08803 2025-05-15 cs.LG cs.AI 73%

Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models

Zizhao Hu, Mohammad Rostami, Jesse Thomason

机构 * University of Southern California（南加州大学）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

2501.00958 2025-05-14 cs.CV cs.CL cs.LG 73%

2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining

Wenqi Zhang, Hang Zhang, Xin Li, Jiashuo Sun, Yongliang Shen, Weiming Lu, Deli Zhao, Yueting Zhuang, Lidong Bing

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）； DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments Under review

2504.17040 2025-05-13 cs.CV cs.AI 73%

DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs

Zhenhailong Wang, Senthil Purushwalkam, Caiming Xiong, Silvio Savarese, Heng Ji, Ran Xu

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Salesforce Research（Salesforce 研究）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2505.03807 2025-05-08 cs.HC cs.AI cs.CV cs.MA 73%

Facilitating Video Story Interaction with Multi-Agent Collaborative System

Yiwen Zhang, Jianing Hao, Zhan Wang, Hongling Sheng, Wei Zeng

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Prepared and submitted in 2024

2505.01694 2025-05-06 cs.CV cs.AI 73%

Topology-Aware CLIP Few-Shot Learning

Dazhi Huang

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2504.21063 2025-05-01 cs.LG cs.AI 73%

Token-Level Prompt Mixture with Parameter-Free Routing for Federated Domain Generalization

Shuai Gong, Chaoran Cui, Xiaolin Dong, Xiushan Nie, Lei Zhu, Xiaojun Chang

机构 * School of Computing and Artificial Intelligence, Shandong University of Finance and Economics（计算机与人工智能学院，山东财经大学）； School of Computer Science and Technology, Shandong Jianzhu University（计算机科学与技术学院，山东建筑大学）； College of Electronics and Information Engineering, Tongji University（电子信息工程学院，同济大学）； School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments The manuscript has been submitted to IEEE Transactions on Knowledge and Data Engineering

2412.09353 2025-04-16 cs.CV cs.AI cs.CL cs.MM 73%

Causal Graphical Models for Vision-Language Compositional Understanding

Fiorenzo Parascandolo, Nicholas Moratelli, Enver Sangineto, Lorenzo Baraldi, Rita Cucchiara

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICLR 2025

2504.00185 2025-04-02 cs.CV cs.LG 73%

Self-Evolving Visual Concept Library using Vision-Language Critics

Atharva Sehgal, Patrick Yuan, Ziniu Hu, Yisong Yue, Jennifer J. Sun, Swarat Chaudhuri

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments CVPR camera ready

2503.09445 2025-04-02 cs.CV cs.AI 73%

Astrea: A MOE-based Visual Understanding Model with Progressive Alignment

Xiaoda Yang, JunYu Lu, Hongshun Qiu, Sijing Li, Hao Li, Shengpeng Ji, Xudong Tang, Jiayang Xu, Jiaqi Duan, Ziyue Jiang, Cong Lin, Sihang Cai, Zejian Xie, Zhuoyang Song, Songxin Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2503.22145 2025-03-31 cs.LG cs.CL cs.CV cs.HC 73%

Tokenization of Gaze Data

Tim Rolff, Jurik Karimian, Niklas Hypki, Susanne Schmidt, Markus Lappe, Frank Steinicke

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

2410.10636 2025-03-25 cs.LG cs.AI 73%

Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection

Adyasha Maharana, Jaehong Yoon, Tianlong Chen, Mohit Bansal

专题命中 VLM训练与架构：grounding(abstract)；multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments First two authors contributed equally. Code: https://github.com/adymaharana/adapt-inf

2503.17416 2025-03-25 cs.SE cs.AI cs.LG 73%

Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)

Boyue Caroline Hu, Divya Gopinath, Corina S. Pasareanu, Nina Narodytska, Ravi Mangal, Susmit Jha

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments CAIN 2025 (4th International Conference on AI Engineering -- Software Engineering for AI)

1. VLM训练与架构 5048 篇

VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data

Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model

Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs

Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehension

From Holistic to Localized: Local Enhanced Adapters for Efficient Visual Instruction Fine-Tuning

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings

Multimodal Prompt Alignment for Facial Expression Recognition

Adding simple structure at inference improves Vision-Language Compositionality

LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning

Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifying

Dual-Process Image Generation

Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts

VITA: Towards Open-Source Interactive Omni Multimodal LLM

QuARI: Query Adaptive Retrieval Improvement

CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling

Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion

MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention

Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models

2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining

DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs

Facilitating Video Story Interaction with Multi-Agent Collaborative System

Topology-Aware CLIP Few-Shot Learning

Token-Level Prompt Mixture with Parameter-Free Routing for Federated Domain Generalization

Causal Graphical Models for Vision-Language Compositional Understanding

Self-Evolving Visual Concept Library using Vision-Language Critics

Astrea: A MOE-based Visual Understanding Model with Progressive Alignment

Tokenization of Gaze Data

Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection

Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)