视觉大模型 / VLM

SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing

SLAM-LLM: 一种模块化、开源的多模态大语言模型框架及语音、语言、音频和音乐处理的最佳实践

Ziyang Ma, Guanrou Yang, Wenxi Chen, Zhifu Gao, Yexing Du, Xiquan Li, Zhisheng Zheng, Haina Zhu, Jianheng Zhuo, Zheshu Song, Ruiyang Xu, Tiranrui Wang, Yifan Yang, Yanqiao Zhu, Zhikang Niu, Liumeng Xue, Yinghao Ma, Ruibin Yuan, Shiliang Zhang, Kai Yu, Eng Siong Chng, Xie Chen

机构 * X-LANCE Lab, School of Computer Science, MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University（X-LANCE实验室，计算机科学学院，人工智能教育部重点实验室，上海交通大学） ； Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团） ； Peng Cheng Laboratory（鹏城实验室） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； Tianjin University（天津大学） ； Hong Kong University of Science and Technology（香港科学大学） ； Queen Mary University of London（伦敦玛丽女王大学） ； Nanyang Technological University（南洋理工大学） ； Shanghai Innovation Institute（上海创新研究院） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)

AI总结 SLAM-LLM是一种开源多模态大语言模型框架，专注于语音、语言、音频和音乐处理，提供模块化配置和高性能检查点以加速研究开发。

Comments Published in IEEE Journal of Selected Topics in Signal Processing (JSTSP)

Focus: A Streaming Concentration Architecture for Efficient Vision-Language Models

聚焦：一种高效的视觉-语言模型流式集中架构

Chiyue Wei, Cong Guo, Junyao Zhang, Haoxuan Shan, Yifan Xu, Ziyue Zhang, Yudong Liu, Qinsi Wang, Changchun Zhou, Hai "Helen" Li, Yiran Chen

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)

AI总结 Focus提出了一种高效的视觉-语言模型流式集中架构，通过分层压缩和细粒度冗余消除，实现2.4倍速度提升和3.3倍能效提升。

Comments HPCA 2026

Adapting Vision-Language Models for Evaluating World Models

为世界模型评估适应视觉-语言模型

Mariya Hendriksen, Tabish Rashid, David Bignell, Raluca Georgescu, Abdelhak Lemkhenter, Katja Hofmann, Sam Devlin, Sarah Parisot

机构 * University of Oxford（牛津大学） ； Microsoft Research（微软研究院） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本文提出UNIVERSE，一种基于视觉-语言模型的视频世界模型评估器，通过适应不同任务格式和数据约束，实现了细粒度、时间敏感的评估，与任务特定检查点性能相当，并在多种环境中验证了其与人类判断的一致性。

Comments NeurIPS LAW 2025 (Oral)

UniFusion: Vision-Language Model as Unified Encoder in Image Generation

Kevin Li, Manuel Brack, Sudeep Katakol, Hareesh Ravi, Ajinkya Kale

机构 * Adobe Applied Research（Adobe应用研究） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project page at https://thekevinli.github.io/unifusion/

Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization

Seongjae Kang, Dong Bok Lee, Hyungjoon Jang, Sung Ju Hwang

机构 * VUNO Inc.（VUNO公司） ； KAIST（韩国科学技术院） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 38 pages, 17 figures, preprint

VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought

Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, Katerina Fragkiadaki

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project website: https://ical-learning.github.io/

Generalized User-Oriented Image Semantic Coding Empowered by Large Vision-Language Model

Sin-Yu Huang, Vincent W. S. Wong

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted by IEEE Global Communications Conference (GLOBECOM), Taipei, Taiwan, Dec. 2025

Fine-Tuning Vision-Language Models for Neutrino Event Analysis in High-Energy Physics Experiments

Dikshant Sagar, Kaiwen Yu, Alejandro Yankelevich, Jianming Bian, Pierre Baldi

机构 * Department of Computer Science University of California, Irvine（计算机科学系加州大学伊文斯顿分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Fine-Tuning Vision-Language Models for Markdown Conversion of Financial Tables in Malaysian Audited Financial Reports

Jin Khye Tan, En Jun Choong, Ethan Jeremiah Chitty, Yan Pheng Choo, John Hsin Yang Wong, Chern Eu Cheah

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 28 pages, 14 figures, 5 tables. Evaluation code (LLM-as-a-judge and Markdown TEDS) is available at https://github.com/jinkhye/MyFinMarkdown. The development dataset and evaluation benchmark are available on Hugging Face at https://huggingface.co/datasets/jinkhye/MyFinMarkdown-sample and https://huggingface.co/datasets/jinkhye/MyFinMarkdown-bench respectively

Adapting Vision-Language Models Without Labels: A Comprehensive Survey

Hao Dong, Lijun Sheng, Jian Liang, Ran He, Eleni Chatzi, Olga Fink

机构 * ETH Zürich（苏黎世联邦理工学院） ； University of Science and Technology of China（中国科学技术大学） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； EPFL（苏黎世联邦理工学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Discussions, comments, and questions are welcome in \url{https://github.com/tim-learn/Awesome-LabelFree-VLMs}

Visual Language Models as Zero-Shot Deepfake Detectors

Viacheslav Pirogov

专题命中 VLM训练与架构 ：visual language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to the ICML 2025 Workshop on Reliable and Responsible Foundation Models

Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embedding

Chetan Pathade

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)

Comments 14 Pages

VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning

Senqiao Yang, Junyi Li, Xin Lai, Bei Yu, Hengshuang Zhao, Jiaya Jia

机构 * CUHK（香港中文大学） ； HKU（香港大学） ； HKUST（香港科技大学） 纠错

专题命中 VLM训练与架构 ：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Code and models are available at https://github.com/dvlab-research/VisionThink

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images

Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber

机构 * Stanford University（斯坦福大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ACL 2025 Main. Project Website: https://s-vco.github.io/

PerSRV: Personalized Sticker Retrieval with Vision-Language Model

Heng Er Metilda Chee, Jiayin Wang, Zhiqiang Guo, Weizhi Ma, Min Zhang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted at WWW '25

Spatial-LLaVA: Enhancing Large Language Models with Spatial Referring Expressions for Visual Understanding

Xuefei Sun, Doncey Albin, Cecilia Mauceri, Dusty Woods, Christoffer Heckman

机构 * Autonomous Robotics and Perception Group in the Computer Science Department at the University of Colorado Boulder（科罗拉多大学波德分校计算机科学系自主机器人与感知小组） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)

A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges

Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, Guangyao Shi

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 22 pages, 3 figures

Journal ref Navigating the Future: Ensuring Trustworthiness in Multi-Modal Open-World Intelligence @ CVPR, 2025

Advancing Chronic Tuberculosis Diagnostics Using Vision-Language Models: A Multi modal Framework for Precision Analysis

Praveen Shastry, Sowmya Chowdary Muthulur, Naveen Kumarasami, Anandakumar D, Mounigasri M, Keerthana R, Kishore Prasath Venkatesh, Bargava Subramanian, Kalyan Sivasailam, Revathi Ezhumalai, Abitha Marimuthu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 10 pages , 3 figures

Advancing vision-language models in front-end development via data synthesis

Tong Ge, Yashu Liu, Jieping Ye, Tianyi Li, Chao Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

How Culturally Aware are Vision-Language Models?

Olena Burda-Lassen, Aman Chadha, Shashank Goswami, Vinija Jain

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Amala Sanjay Deshmukh, Tuomas Rintamaki, Matthieu Le, Ilia Karmanov, Lukas Voegtle, Philipp Fischer, De-An Huang, Timo Roman, Tong Lu, Jose M. Alvarez, Bryan Catanzaro, Jan Kautz, Andrew Tao, Guilin Liu, Zhiding Yu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Visual RAG: Expanding MLLM visual knowledge without fine-tuning

Mirco Bonomo, Simone Bianco

专题命中 VLM训练与架构 ：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

PaliGemma: A versatile 3B VLM for transfer

Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, Xiaohua Zhai

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments v2 adds Appendix H and I and a few citations

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, Sergey Levine

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

SQ-LLaVA: Self-Questioning for Large Vision-Language Assistant

Guohao Sun, Can Qin, Jiamian Wang, Zeyuan Chen, Ran Xu, Zhiqiang Tao

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ECCV 2024

Visual Language Model based Cross-modal Semantic Communication Systems

Feibo Jiang, Chuanguo Tang, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan

专题命中 VLM训练与架构 ：visual language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 12 pages, 10 figures

Ovis: Structural Embedding Alignment for Multimodal Large Language Model

Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Han-Jia Ye

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models

Christian Schlarmann, Naman Deep Singh, Francesco Croce, Matthias Hein

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2024 Oral

Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research

Qinglong Cao, Yuntian Chen, Lu Lu, Hao Sun, Zhenzhong Zeng, Xiaokang Yang, Dongxiao Zhang

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

BRAVE: Broadening the visual encoding of vision-language models

Oğuzhan Fatih Kar, Alessio Tonioni, Petra Poklukar, Achin Kulshrestha, Amir Zamir, Federico Tombari

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project page at https://brave-vlms.epfl.ch/