视觉大模型 / VLM

BiPVL-Seg: Bidirectional Progressive Vision-Language Fusion with Global-Local Alignment for Medical Image Segmentation

Rafi Ibn Sultan, Hui Zhu, Chengyin Li, Dongxiao Zhu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

LongViTU: Instruction Tuning for Long-Form Video Understanding

Rujie Wu, Xiaojian Ma, Hai Ci, Yue Fan, Yuxuan Wang, Haozhe Zhao, Qing Li, Yizhou Wang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.LG

From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignment

Yucheng Suo, Fan Ma, Linchao Zhu, Tianyi Wang, Fengyun Rao, Yi Yang

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.AI

Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis

Letian Zhang, Quan Cui, Bingchen Zhao, Cheng Yang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Data Augmentation in Earth Observation: A Diffusion Model Approach

Tiago Sousa, Benoît Ries, Nicolas Guelfi

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 25 pages, 12 figures

Journal ref Information 2025, 16, 81

Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations

Jeonghyeon Kim, Sangheum Hwang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding

Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zenghui Ding, Xianjun Yang, Yining Sun

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Code is available at https://github.com/Jam1ezhang/DYTO

JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation

Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai yu, Liang Zhao, Yisong Wang, Jiaying Liu, Chong Ruan

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2025

Wolf: Dense Video Captioning with a World Summarization Framework

Boyi Li, Ligeng Zhu, Ran Tian, Shuhan Tan, Yuxiao Chen, Yao Lu, Yin Cui, Sushant Veer, Max Ehrlich, Jonah Philion, Xinshuo Weng, Fuzhao Xue, Linxi Fan, Yuke Zhu, Jan Kautz, Andrew Tao, Ming-Yu Liu, Sanja Fidler, Boris Ivanovic, Trevor Darrell, Jitendra Malik, Song Han, Marco Pavone

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.LG

Prompt Injection Attacks on Large Language Models in Oncology

Jan Clusmann, Dyke Ferber, Isabella C. Wiest, Carolin V. Schneider, Titus J. Brinker, Sebastian Foersch, Daniel Truhn, Jakob N. Kather

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.AI、cs.LG

Comments 57 Pages, 5 Figures

CaReBench: A Fine-Grained Benchmark for Video Captioning and Retrieval

Yifan Xu, Xinhao Li, Yichun Yang, Desen Meng, Rui Huang, Limin Wang

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.LG

TacticExpert: Spatial-Temporal Graph Language Model for Basketball Tactics

Xu Lingrui, Liu Mandi, Zhang Lei

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.AI、cs.LG

A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosis

Xiang Liu, Zhaoxiang Liu, Huan Hu, Zezhou Chen, Kohou Wang, Kai Wang, Shiguo Lian

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV 2024 (14 pages, 8 figures)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models

Yongxin Guo, Zhenglin Cheng, Xiaoying Tang, Zhaopeng Tu, Tao Lin

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2025

VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, Song Han, Yao Lu

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.LG

Comments Code: https://github.com/mit-han-lab/vila-u. The first two authors contributed equally to this work

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models

Makoto Shing, Kou Misaki, Han Bao, Sho Yokoi, Takuya Akiba

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.AI、cs.LG

Comments To appear at the 13th International Conference on Learning Representations (ICLR 2025) as a Spotlight presentation

SAN: Hypothesizing Long-Term Synaptic Development and Neural Engram Mechanism in Scalable Model's Parameter-Efficient Fine-Tuning

Gaole Dai, Chun-Kai Fan, Yiming Tang, Zhi Zhang, Yuan Zhang, Yulu Gan, Qizhe Zhang, Cheng-Ching Tseng, Shanghang Zhang, Tiejun Huang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI、cs.LG

Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning

Weitai Kang, Haifeng Huang, Yuzhang Shang, Mubarak Shah, Yan Yan

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments 8 pages

MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding

Zekun Li, Xianjun Yang, Kyuri Choi, Wanrong Zhu, Ryan Hsieh, HyeonJung Kim, Jin Hyuk Lim, Sungyoung Ji, Byungju Lee, Xifeng Yan, Linda Ruth Petzold, Stephen D. Wilson, Woosang Lim, William Yang Wang

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI

Comments Code and data are available at https://github.com/Leezekun/MMSci

Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning

Hao Ma, Shijie Wang, Zhiqiang Pu, Siyao Zhao, Xiaolin Ai

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.AI、cs.LG

Intuitive physics understanding emerges from self-supervised pretraining on natural videos

Quentin Garrido, Nicolas Ballas, Mahmoud Assran, Adrien Bardes, Laurent Najman, Michael Rabbat, Emmanuel Dupoux, Yann LeCun

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 24 pages,14 figures, 5 tables

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training

Haocheng Xi, Han Cai, Ligeng Zhu, Yao Lu, Kurt Keutzer, Jianfei Chen, Song Han

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by ICLR 2025. 22 pages. 9 Figures. 13 Tables

mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Information Theoretic Text-to-Image Alignment

Chao Wang, Giulio Franzese, Alessandro Finamore, Massimo Gallo, Pietro Michiardi

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments to appear at ICLR25

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

Greg Heinrich, Mike Ranzinger, Hongxu, Yin, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, Pavlo Molchanov

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI

GRAPE: Generalizing Robot Policy via Preference Alignment

Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox, Huaxiu Yao

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Website: https://grape-vla.github.io/

Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification

Xiangyu Sun, Xiaoguang Zou, Yuanquan Wu, Guotai Wang, Shaoting Zhang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments This paper has been accepted for presentation at the 2025 IEEE International Symposium on Biomedical Imaging (ISBI 2025)

PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures

Shreya Shukla, Nakul Sharma, Manish Gupta, Anand Mishra

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2025 (Main Track). Project page: https://vl2g.github.io/projects/PatentLMM/

From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events

Yan Miao, Georgios Fainekos, Bardh Hoxha, Hideki Okamoto, Danil Prokhorov, Sayan Mitra

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Parameter-Efficient Fine-Tuning for Foundation Models

Dan Zhang, Tao Feng, Lilong Xue, Yuandong Wang, Yuxiao Dong, Jie Tang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 6 figures, 7 tables