视觉大模型 / VLM

An Experimental Study on Generating Plausible Textual Explanations for Video Summarization

Thomas Eleftheriadis, Evlampios Apostolidis, Vasileios Mezaris

机构 * IEEE CBMI 2025 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments IEEE CBMI 2025. This is the authors' accepted version. The final publication is available at https://ieeexplore.ieee.org/

Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization

Xintong Li, Chuhan Wang, Junda Wu, Rohan Surana, Tong Yu, Julian McAuley, Jingbo Shang

机构 * University of California, San Diego（加州大学圣迭戈分校） ； Adobe Research（Adobe研究） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Preprint

Patient-specific Biomolecular Instruction Tuning

Irsyad Adam, Zekai Chen, David Laub, Shaun Porwal, Arda Pekis, Kevin Brown

机构 * Standard Model Biomedicine（标准模型生物医学） ； University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Random Direct Preference Optimization for Radiography Report Generation

Valentin Samokhin, Boris Shirokikh, Mikhail Goncharov, Dmitriy Umerenkov, Maksim Bobrin, Ivan Oseledets, Dmitry Dylov, Mikhail Belyaev

机构 * Artificial Intelligence Research Institute (AIRI)（人工智能研究院） ； Skolkovo Institute of Science and Technology（斯克罗夫学院） ； Institute for Information Transmission Problems（信息传输问题研究所） 纠错

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.AI

Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models

Zoe Wanying He, Sean Trott, Meenakshi Khosla

机构 * Department of Cognitive Science University of California, San Diego（认知科学系，加州大学圣地亚哥分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at EMNLP 2025 (camera-ready)

AdaSVD: Adaptive Singular Value Decomposition for Large Language Models

Zhiteng Li, Mingyuan Xia, Jingyuan Zhang, Zheng Hui, Haotong Qin, Linghe Kong, Yulun Zhang, Xiaokang Yang

机构 * Shanghai Jiao Tong University（上海交通大学） ； MGTV, Shanhai Academy（MGTV，汕海学院） ； ETH Zürich（苏黎世联邦理工学院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments The code and models will be available at https://github.com/ZHITENGLI/AdaSVD

Exploration with Foundation Models: Capabilities, Limitations, and Hybrid Approaches

Remo Sasso, Michelangelo Conserva, Dominik Jeurissen, Paulo Rauber

机构 * School of Electronic Engineering and Computer Science（电子工程与计算机科学学院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.AI、cs.LG

Comments 16 pages, 7 figures. Accepted for presentation at the 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop on the Foundations of Reasoning in Language Models (FoRLM)

Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity

Zhaoyi Joey Hou, Adriana Kovashka, Xiang Lorraine Li

机构 * Department of Computer Science University of Pittsburgh（计算机科学系宾夕法尼亚大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI

Comments To Appear in EMNLP2025

Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning

Riad Ahmed Anonto, Sardar Md. Saffat Zabin, M. Saifur Rahman

机构 * Bangladesh University of Engineering and Technology (BUET)（孟加拉工程与技术大学） 纠错

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV、cs.AI

Multimodal Health Risk Prediction System for Chronic Diseases via Vision-Language Fusion and Large Language Models

Dingxin Lu, Shurui Wu, Xinyi Huang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI、cs.LG

MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe

Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, Bokai Xu, Junbo Cui, Yingjing Xu, Liqing Ruan, Luoyuan Zhang, Hanyu Liu, Jingkun Tang, Hongyuan Liu, Qining Guo, Wenhao Hu, Bingxiang He, Jie Zhou, Jie Cai, Ji Qi, Zonghao Guo, Chi Chen, Guoyang Zeng, Yuxuan Li, Ganqu Cui, Ning Ding, Xu Han, Yuan Yao, Zhiyuan Liu, Maosong Sun

机构 * MiniCPM-V Team, OpenBMB（MiniCPM-V团队，OpenBMB） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Project Website: https://github.com/OpenBMB/MiniCPM-V

MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer

Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang, Haoshuo Huang, Bowen Zhang, Jialing Tong, Haoxuan You, Xianzhi Du, Zhe Gan, Hyunjik Kim, Chao Jia, Zhenbang Wang, Yinfei Yang, Mingfei Gao, Zi-Yi Dou, Wenze Hu, Chang Gao, Dongxu Li, Philipp Dufter, Zirui Wang, Guoli Yin, Zhengdong Zhang, Chen Chen, Yang Zhao, Ruoming Pang, Zhifeng Chen

机构 * Apple（苹果公司） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Discrete Diffusion in Large Language and Multimodal Models: A Survey

Runpeng Yu, Qi Li, Xinchao Wang

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.AI、cs.LG

DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing

Xiaolong Wang, Zhi-Qi Cheng, Jue Wang, Xiaojiang Peng

机构 * Shenzhen Technology University（深圳科技大学） ； University of Washington（华盛顿大学） ； Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 13 pages,12 figures

Recurrence Meets Transformers for Universal Multimodal Retrieval

Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

机构 * Department of Education and Humanities, University of Modena and Reggio Emilia（教育与人文学院, Modena and Reggio Emilia大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Image Embedding Sampling Method for Diverse Captioning

Sania Waheed, Na Min An

机构 * University of Southhampton（南安普顿大学） ； KAIST（韩国科学技术院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments 17 pages, 5 figures, 9 tables

GAEA: A Geolocation Aware Conversational Assistant

Ron Campos, Ashmal Vayani, Parth Parag Kulkarni, Rohit Gupta, Aizan Zafar, Aritra Dutta, Mubarak Shah

机构 * Center for Research in Computer Vision, University of Central Florida（计算机视觉研究中心，中央佛罗里达大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.LG

Comments The dataset and code used in this submission is available at: https://ucf-crcv.github.io/GAEA/

Fusion to Enhance: Fusion Visual Encoder to Enhance Multimodal Language Model

Yifei She, Huangxuan Wu

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning

Yicheng Ji, Jun Zhang, Heming Xia, Jinpeng Chen, Lidan Shou, Gang Chen, Huan Li

机构 * The State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室） ； Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院） ； Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系） ； School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted at EMNLP 2025 Main

Real-Time Intuitive AI Drawing System for Collaboration: Enhancing Human Creativity through Formal and Contextual Intent Integration

Jookyung Song, Mookyoung Kang, Nojun Kwak

机构 * SNU（首尔国立大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 4 figures, NeurIPS Creative AI Track 2025

VISA: Group-wise Visual Token Selection and Aggregation via Graph Summarization for Efficient MLLMs Inference

Pengfei Jiang, Hanjun Li, Linglan Zhao, Fei Chao, Ke Yan, Shouhong Ding, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室，教育部，厦门大学） ； Tencent Youtu Lab（腾讯优图实验室） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACMMM 2025

$Δ$-AttnMask: Attention-Guided Masked Hidden States for Efficient Data Selection and Augmentation

Jucheng Hu, Suorong Yang, Dongzhan Zhou

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions

Moran Yanuka, Assaf Ben Kish, Yonatan Bitton, Idan Szpektor, Raja Giryes

机构 * Tel Aviv University（特拉维夫大学） ； Google Research（谷歌研究） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NAACL 2025

Journal ref Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics, Human Language Technologies, Long Papers, pp. 10497-10518

Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning

Chen Shen, Chunfeng Lian, Wanqing Zhang, Fan Wang, Jianhua Zhang, Shuanliang Fan, Xin Wei, Gongji Wang, Kehan Li, Hongshu Mu, Hao Wu, Xinggong Liang, Jianhua Ma, Zhenyuan Wang

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments 28 pages, 6 figures, under review

Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents

Han Lin, Jaemin Cho, Amir Zadeh, Chuan Li, Mohit Bansal

机构 * UNC Chapel Hill（北卡罗来纳大学教堂山分校） ； Lambda 纠错

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://bifrost-1.github.io

TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation

Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K. Du, Zehuan Yuan, Xinglong Wu

机构 * ByteDance（字节跳动） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025; Code and models: https://github.com/ByteVisionLab/TokenFlow

True Multimodal In-Context Learning Needs Attention to the Visual Context

Shuo Chen, Jianzhe Liu, Zhen Han, Yan Xia, Daniel Cremers, Philip Torr, Volker Tresp, Jindong Gu

机构 * LMU Munich（慕尼黑大学） ； Technical University of Munich（慕尼黑技术大学） ； Siemens AG（西门子股份公司） ； University of Science and Technology of China（中国科学技术大学） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心） ； Konrad Zuse School of Excellence in Reliable AI (relAI)（Konrad Zuse可靠性人工智能卓越学院） ； University of Oxford（牛津大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to COLM 2025

CLIPTime: Time-Aware Multimodal Representation Learning from Images and Text

Anju Rani, Daniel Ortiz-Arroyo, Petar Durdevic

机构 * Department of Energy Technology（能源技术系） ； Aalborg University（奥尔堡大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments 11 pages, 8 figures

Decouple before Align: Visual Disentanglement Enhances Prompt Tuning

Fei Zhang, Tianfei Zhou, Jiangchao Yao, Ya Zhang, Ivor W. Tsang, Yanfeng Wang

机构 * Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（合作中位网创新中心，上海交通大学） ； School of Artificial Intelligence, Shanghai Jiao Tong University（人工智能学院，上海交通大学） ； Shanghai Innovation Institute（上海创新研究院） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Beijing Institute of Technology（北京理工大学） ； A*STAR Centre for Frontier AI Research（A*STAR前沿人工智能研究中心） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, Accepted at IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product

Paul Albert, Frederic Z. Zhang, Hemanth Saratchandran, Anton van den Hengel, Ehsan Abbasnejad

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments To appear in ICCV 2025