视觉大模型 / VLM

What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation

Michal Golovanevsky, William Rudman, Vedant Palit, Ritambhara Singh, Carsten Eickhoff

机构 * Department of Computer Science, Brown University（布朗大学计算机科学系） ； Indian Institute of Technology Kharagpur（印度理工学院Khargpur分校） ； Center for Computational Molecular Biology, Brown University（布朗大学计算分子生物学中心） ； School of Medicine, University of Tübingen（图宾根大学医学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

MA-ROESL: Motion-aware Rapid Reward Optimization for Efficient Robot Skill Learning from Single Videos

Xianghui Wang, Xinming Zhang, Yanjun Chen, Xiaoyu Shen, Wei Zhang

机构 * College of Information Science and Technology, Eastern Institute of Technology（信息科学与技术学院，东部技术学院） ； School of Computer Science and Technology, The University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学） ； Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

UAV-VLRR: Vision-Language Informed NMPC for Rapid Response in UAV Search and Rescue

Yasheerah Yaqoot, Muhammad Ahsan Mustafa, Oleg Sautenkov, Artem Lykov, Valerii Serpiva, Dzmitry Tsetserukou

机构 * Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology（智能空间机器人实验室、数字工程中心、斯克尔科沃科学与技术研究所） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)

Comments UAV-VLRR

An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understanding

Siyang Jiang, Bufang Yang, Lilin Xu, Mu Yuan, Yeerzhati Abudunuer, Kaiwei Liu, Liekang Zeng, Hongkai Chen, Zhenyu Yan, Xiaofan Jiang, Guoliang Xing

机构 * The Chinese University of Hong Kong, Hong Kong SAR（香港中文大学） ； Columbia University, United States（哥伦比亚大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?

Letitia Parcalabescu, Anette Frank

机构 * Computational Linguistics Department（计算语言学系） ； Heidelberg University（海德堡大学） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages, 8 figures, 11 tables

Can VLMs Assess Similarity Between Graph Visualizations?

Seokweon Jung, Hyeon Jeon, Jeongmin Rhee, Jinwook Seo

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)

TULIP: Towards Unified Language-Image Pretraining

Zineng Tang, Long Lian, Seun Eisape, XuDong Wang, Roei Herzig, Adam Yala, Alane Suhr, Trevor Darrell, David M. Chan

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments (v2) Clarified fine-tuning process, updated appendix

Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources

Weizhi Wang, Yu Tian, Linjie Yang, Heng Wang, Xifeng Yan

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)

Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck

Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation

Sankalp Sinha, Mohammad Sadil Khan, Muhammad Usama, Shino Sam, Didier Stricker, Sk Aziz Ali, Muhammad Zeshan Afzal

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

RankCLIP: Ranking-Consistent Language-Image Pretraining

Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zhili Feng, Zenghui Ding, Yining Sun

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Code and model checkpoints are available at https://github.com/Jam1ezhang/RankCLIP

Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)

Leander Girrbach, Stephan Alaniz, Yiran Huang, Trevor Darrell, Zeynep Akata

专题命中 VLM训练与架构 ：LLaVA(abstract)；InternVL(abstract)

Comments Accepted at ICLR 2025

COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection

Jinqi Xiao, Shen Sang, Tiancheng Zhi, Jing Liu, Qing Yan, Yuqian Zhang, Linjie Luo, Bo Yuan

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2025

Trinity: A Modular Humanoid Robot AI System

Jingkai Sun, Qiang Zhang, Gang Han, Wen Zhao, Zhe Yong, Yan He, Jiaxu Wang, Jiahang Cao, Yijie Guo, Renjing Xu

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)

LVPruning: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Models

Yizheng Sun, Yanze Xin, Hao Li, Jingyuan Sun, Chenghua Lin, Riza Batista-Navarro

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

Comments Accepted to NAACL 2025 Findings

Can Large Language Models Grasp Concepts in Visual Content? A Case Study on YouTube Shorts about Depression

Jiaying "Lizzy" Liu, Yiheng Su, Praneel Seth

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

Comments 11 pages

Journal ref CHI Conference on Human Factors in Computing Systems (CHI EA 2025)

Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs

Wei Zhao, Zhe Li, Yige Li, Jun Sun

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Training-free Diffusion Model Alignment with Sampling Demons

Po-Hung Yeh, Kuang-Huei Lee, Jun-Cheng Chen

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 35 pages

Journal ref Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025)

NOTA: Multimodal Music Notation Understanding for Visual Large Language Model

Mingni Tang, Jiajia Li, Lu Yang, Zhiqiang Zhang, Jinghao Tian, Zuchao Li, Lefei Zhang, Ping Wang

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Routing Experts: Learning to Route Dynamic Experts in Multi-modal Large Language Models

Qiong Wu, Zhaoxi Ke, Yiyi Zhou, Xiaoshuai Sun, Rongrong Ji

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

EnVisionVR: A Scene Interpretation Tool for Visual Accessibility in Virtual Reality

Junlong Chen, Rosella P. Galindo Esparza, Vanja Garaj, Per Ola Kristensson, John Dudley

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)

Comments The online appendix is available at https://osf.io/zb2ak/

OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints

Mingjie Pan, Jiyao Zhang, Tianshu Wu, Yinghao Zhao, Wenlong Gao, Hao Dong

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data

Shuhao Gu, Jialing Zhang, Siyuan Zhou, Kevin Yu, Zhaohu Xing, Liangdong Wang, Zhou Cao, Jintao Jia, Zhuoyi Zhang, Yixuan Wang, Zhenchong Hu, Bo-Wen Zhang, Jijie Li, Dong Liang, Yingli Zhao, Songjing Wang, Yulong Ao, Yiming Ju, Huanhuan Ma, Xiaotong Li, Haiwen Diao, Yufeng Cui, Xinlong Wang, Yaoqi Liu, Fangxiang Feng, Guang Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

RealMind: Advancing Visual Decoding and Language Interaction via EEG Signals

Dongyang Li, Haoyang Qin, Mingyang Wu, Jiahua Tang, Yuang Cao, Chen Wei, Quanying Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

MMFactory: A Universal Solution Search Engine for Vision-Language Tasks

Wan-Cyuan Fan, Tanzila Rahman, Leonid Sigal

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning

Yuti Liu, Shice Liu, Junyuan Gao, Pengtao Jiang, Hao Zhang, Jinwei Chen, Bo Li

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by AAAI 2025

Visual Lexicon: Rich Image Features in Language Space

XuDong Wang, Xingyi Zhou, Alireza Fathi, Trevor Darrell, Cordelia Schmid

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Tech report. 16 pages, 10 figures

Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen, Chengxu Zhuang, Ryan Cotterell, Leshem Choshen, Alex Warstadt, Ethan Gotlieb Wilcox

专题命中 VLM训练与架构 ：vision language model(abstract)；grounding(abstract)

BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding

Chenguang Huang, Shengchao Yan, Wolfram Burgard

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge

Yaqi Zhao, Yuanyang Yin, Lin Li, Mingan Lin, Victor Shea-Jay Huang, Siwei Chen, Weipeng Chen, Baoqun Yin, Zenan Zhou, Wentao Zhang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG