视觉大模型 / VLM

MATE: Meet At The Embedding -- Connecting Images with Long Texts

Young Kyun Jang, Junmo Kang, Yong Jae Lee, Donghyun Kim

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?

Zhaorun Chen, Yichao Du, Zichen Wen, Yiyang Zhou, Chenhang Cui, Zhenzhen Weng, Haoqin Tu, Chaoqi Wang, Zhengwei Tong, Qinglan Huang, Canyu Chen, Qinghao Ye, Zhihong Zhu, Yuqing Zhang, Jiawei Zhou, Zhuokai Zhao, Rafael Rafailov, Chelsea Finn, Huaxiu Yao

专题命中 VLM训练与架构 ：VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments 42 pages, 13 figures, 33 tables

RAVEN: Multitask Retrieval Augmented Vision-Language Learning

Varun Nagaraj Rao, Siddharth Choudhary, Aditya Deshpande, Ravi Kumar Satzoda, Srikar Appalaraju

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

MammothModa: Multi-Modal Large Language Model

Qi She, Junwen Pan, Xin Wan, Rui Zhang, Dawei Lu, Kai Huang

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Technical report

Directed Domain Fine-Tuning: Tailoring Separate Modalities for Specific Training Tasks

Daniel Wen, Nafisa Hussain

专题命中 VLM训练与架构 ：visual language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression

Zilun Zhang, Yutao Sun, Tiancheng Zhao, Leigang Sha, Ruochen Xu, Kyusong Lee, Jianwei Yin

专题命中 VLM训练与架构 ：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Multimodal LLMs Struggle with Basic Visual Network Analysis: a VNA Benchmark

Evan M. Williams, Kathleen M. Carley

专题命中 VLM训练与架构 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 3 figures

OLIVE: Object Level In-Context Visual Embeddings

Timothy Ossowski, Junjie Hu

专题命中 VLM训练与架构 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments ACL 2024

VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation

Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, Wenhu Chen

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACL2024 main

UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

ShengYun Peng, Aishwarya Chakravarthy, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts

Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, Min Zhang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 22 pages, 13 figures. Project Website: https://uni-moe.github.io/. Working in progress

UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark

Zhaokun Zhou, Qiulin Wang, Bin Lin, Yiwei Su, Rui Chen, Xin Tao, Amin Zheng, Li Yuan, Pengfei Wan, Di Zhang

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labeling

Hritik Bansal, Po-Nien Kung, P. Jeffrey Brantingham, Kai-Wei Chang, Nanyun Peng

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 20 pages, 15 Figures, 13 figures

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization

Kailin Li, Jingbo Wang, Lixin Yang, Cewu Lu, Bo Dai

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

Chull Hwan Song, Taebaek Hwang, Jooyoung Yoon, Shunghyun Choi, Yeong Hyeon Gu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments CVPR2024 Accepted

Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation

Zhiwei Zhang, Yuliang Liu

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

Comments TMLR, Survey Certification

SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment

Ziping Ma, Furong Xu, Jian Liu, Ming Yang, Qingpei Guo

专题命中 VLM训练与架构 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Text Descriptions are Compressive and Invariant Representations for Visual Learning

Zhili Feng, Anna Bair, J. Zico Kolter

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Resolving References in Visually-Grounded Dialogue via Text Generation

Bram Willemsen, Livia Qian, Gabriel Skantze

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Published at SIGDIAL 2023

Language Reward Modulation for Pretraining Reinforcement Learning

Ademi Adeniji, Amber Xie, Carmelo Sferrazza, Younggyo Seo, Stephen James, Pieter Abbeel

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments Code available at https://github.com/ademiadeniji/lamp

MIMIC-IT: Multi-Modal In-Context Instruction Tuning

Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, Ziwei Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://otter-ntu.github.io/ Dataset & code: https://github.com/Luodian/otter Initial release, work in progress

Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment

Mustafa Shukor, Guillaume Couairon, Matthieu Cord

专题命中 VLM训练与架构 ：visual reasoning(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments BMVC 2022

Deep Multimodal Neural Architecture Search

Zhou Yu, Yuhao Cui, Jun Yu, Meng Wang, Dacheng Tao, Qi Tian

专题命中 VLM训练与架构 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Accept to ACM MM2020, code available at https://github.com/MILVLG/mmnas/

Token Warping Helps MLLMs Look from Nearby Viewpoints

令牌扭曲帮助多模态大语言模型从近处视角观察

Phillip Y. Lee, Chanho Park, Mingue Park, Seungwoo Yoo, Juil Koo, Minhyuk Sung

机构 * KAIST（韩国科学技术院） 纠错

专题命中 VLM训练与架构 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV；MLLM(comments)

AI总结 本文研究令牌扭曲如何帮助多模态大语言模型从近处视角理解场景，发现反向令牌扭曲在视角变化中更稳定且能更好保持语义一致性。

Comments CVPR 2026, Project Page: https://token-warping-mllm.github.io

Redundancy-Aware Pretraining of Vision-Language Foundation Models in Remote Sensing

Mathis Jürgen Adler, Leonard Hackel, Gencer Sumbul, Begüm Demir

机构 * TU Berlin（柏林技术大学） ； BIFOLD（BIFOLD机构） ； EPFL（苏黎世联邦理工学院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,comments)；vision-language model(abstract)；分类 cs.CV

Comments Accepted at IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2025. Our code is available at https://git.tu-berlin.de/rsim/redundacy-aware-rs-vlm

Training-Free Multimodal Large Language Model Orchestration

免训练的多模态大语言模型编排

Tianyu Xie, Yuexiao Ma, Yuhang Wu, Wang Chen, Jiayi Ji, Tat-Seng Chua, Xiawu Zheng, Rongrong Ji

机构 * Media Analytics and Computing Lab, Xiamen University, Xiamen, China（厦门大学媒体分析与计算实验室） ； Institute of Artificial Intelligence, Xiamen University, Xiamen, China（厦门大学人工智能研究院） ； School of Informatics, Xiamen University, Xiamen, China（厦门大学信息学院） ； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室） ； Department of Computer Science, National University of Singapore, Singapore（新加坡国立大学计算机科学系） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title)

AI总结 提出一种免训练编排框架LLM Orchestration，通过LLM控制器、文本中心跨模态记忆和统一交互层集成现成模态专家，实现低开销、可扩展的多模态输入输出系统。

Journal ref ICML 2026

Quantification and object perception in Multimodal Large Language Models and human linguistic cognition

多模态大语言模型和人类语言认知中的量化与物体感知

Raquel Montero, Natalia Moskvina, Paolo Morosi, Tamara Serrano, Elena Pagliarini, Evelina Leivada

机构 * Universitat Autònoma de Barcelona（巴塞罗那自治大学） ； Institució Catalana de Recerca i Estudis Avançats (ICREA)（加泰罗尼亚高级科研与研究机构（ICREA）） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title)

AI总结 本文研究多模态大语言模型和人类在量化中的关键特征，探讨其在模型架构中的编码差异及语言影响。

Speech-Omni-Lite: Portable Speech Interfaces for Vision-Language Models

Speech-Omni-Lite: 用于视觉-语言模型的便携式语音接口

Dehua Tao, Xuan Luo, Daxin Tan, Kai Chen, Lanqing Hong, Jing Li, Ruifeng Xu, Xiao Chen

专题命中 VLM训练与架构 ：vision-language model(title)

AI总结 Speech-Omni-Lite通过轻量模块扩展视觉-语言模型，实现低成本语音理解和生成，同时保持原有性能，实验显示其在少量语音数据下表现优异。

SGPA: Spectrogram-Guided Phonetic Alignment for Feasible Shapley Value Explanations in Multimodal Large Language Models

SGPA: 基于频谱的语音对齐用于多模态大语言模型中可行的谢普利值解释

Paweł Pozorski, Jakub Muszyński, Maria Ganzha

机构 * Warsaw University of Technology（华沙技术大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title)

AI总结 SGPA通过结合连接主义时间分类和频谱边界细化，实现了多模态大语言模型中可行的音频解释，显著减少了模型评估次数并保持了全局轮廓。

Comments Submitted for admission in Interspeech 2026 conference

Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Driving

Sim2Real Diffusion：利用基础视觉语言模型实现自适应自动驾驶

Chinmay Vilas Samak, Tanmay Vilas Samak, Bing Li, Venkat Krovi

机构 * Department of Automotive Engineering, Clemson University International Center for Automotive Research (CU-ICAR)（汽车工程系，克莱姆森大学国际汽车研究中心（CU-ICAR）） 纠错

专题命中 VLM训练与架构 ：vision language model(title)

AI总结 本文提出Sim2Real Diffusion框架，利用基础视觉语言模型实现自动驾驶的跨领域适应，通过条件潜在扩散提升sim2real转换性能。

Comments Accepted in IEEE Robotics and Automation Letters (RA-L)

Journal ref IEEE Robotics and Automation Letters, vol. 11, no. 1, pp. 177-184, Jan. 2026