视觉大模型 / VLM

Cross-Modal Adapter: Parameter-Efficient Transfer Learning Approach for Vision-Language Models

Juncheng Yang, Zuchao Li, Shuai Xie, Weiping Zhu, Wei Yu, Shijun Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments This paper is accepted to ICME 2024

DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback

Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, Ajay Divakaran

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；分类 cs.CV、cs.LG

Comments CVPR 2024. The feedback datasets are released at: https://huggingface.co/datasets/YangyiYY/LVLM_NLF

LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images

Ruyi Xu, Yuan Yao, Zonghao Guo, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, Maosong Sun, Gao Huang

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV、cs.AI

Comments Preprint

Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models

Vishaal Udandarao, Max F. Burg, Samuel Albanie, Matthias Bethge

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Semantic-Aware Frame-Event Fusion based Pattern Recognition via Large Vision-Language Models

Dong Li, Jiandong Jin, Yuhao Zhang, Yanlin Zhong, Yaoyang Wu, Lan Chen, Xiao Wang, Bin Luo

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare

Junling Liu, Ziming Wang, Qichen Ye, Dading Chong, Peilin Zhou, Yining Hua

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Class Incremental Learning with Pre-trained Vision-Language Models

Xialei Liu, Xusheng Cao, Haori Lu, Jia-wen Xiao, Andrew D. Bagdanov, Ming-Ming Cheng

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph

Xin Li, Dongze Lian, Zhihe Lu, Jiawang Bai, Zhibo Chen, Xinchao Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2023. The manuscript will be further revised based on the reviews

InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments preprint

PuMer: Pruning and Merging Tokens for Efficient Vision Language Models

Qingqing Cao, Bhargavi Paranjape, Hannaneh Hajishirzi

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACL 2023 Main Conference

You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model

Shengkun Tang, Yaqing Wang, Zhenglun Kong, Tianchi Zhang, Yao Li, Caiwen Ding, Yanzhi Wang, Yi Liang, Dongkuan Xu

专题命中 VLM训练与架构 ：vision language model(title)；visual language model(abstract)；分类 cs.CV、cs.LG

Prompt Tuning based Adapter for Vision-Language Model Adaption

Jingchen Sun, Jiayu Qin, Zihao Lin, Changyou Chen

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

A Multi-level Alignment Training Scheme for Video-and-Language Grounding

Yubo Zhang, Feiyang Niu, Qing Ping, Govind Thattai

专题命中 VLM训练与架构 ：grounding(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted at ICDM 2022 FOMO-VL workshop

Journal ref 2022 IEEE International Conference on Data Mining Workshops (ICDMW), Orlando, FL, USA, 2022, pp. 958-966

Localized Latent Updates for Fine-Tuning Vision-Language Models

Moritz Ibing, Isaak Lim, Leif Kobbelt

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Towards Multimodal Vision-Language Models Generating Non-Generic Text

Wes Robbins, Zanyar Zohourianshahzadi, Jugal Kalita

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Journal ref 2021 International Conference on Natural Language Processing

Compositional Context Fine-Tuning Vision-Language Model for Complex Assembly Action Understanding from Videos

用于从视频理解复杂装配动作的组合上下文微调视觉语言模型

Hao Zheng, Jinyi Huang, Tiantian Zheng, Xun Xu, Tuka Alhanai

机构 * New York University Abu Dhabi（纽约大学阿布扎比分校） ； The University of Auckland（奥克兰大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV；vision language model(comments)

AI总结 针对装配动作理解难题，提出组合上下文微调方法及层划分交替训练方法，将动作分解为语义元素并微调视觉语言模型，创建相关数据集，实验证明该方法优于基线且能提供可解释预测，助力人机协作装配。

Comments Accepted by ICRA 2026. Video Understanding; Vision Language Model; Multi-modal LLM; Action Recognition; Assembly

Vision Language Models: A Survey of 26K Papers

Fengming Lin

机构 * School of Computer Science, The University of Manchester, Manchester, UK（曼彻斯特大学计算机科学学院） 纠错

专题命中 VLM训练与架构 ：vision language model(title)；VLM(abstract,comments)；分类 cs.CV

Comments VLM/LLM Learning Notes

LLaVA-Video: Video Instruction Tuning With Synthetic Data

Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, Chunyuan Li

机构 * S-Lab, Nanyang Technological University（南洋理工大学S实验室） ； BUPT（北京邮电大学） ； ByteDance（字节跳动） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Project page: https://llava-vl.github.io/blog/2024-09-30-llava-video/; Accepted at TMLR

LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness

Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, Xihui Liu

机构 * The University of Hong Kong（香港大学） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Project page: https://zcmax.github.io/projects/LLaVA-3D/

LLaVA-Critic: Learning to Evaluate Multimodal Models

Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, Chunyuan Li

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2025; Project Page: https://llava-vl.github.io/blog/2024-10-03-llava-critic

Natural Language Inference Improves Compositionality in Vision-Language Models

Paola Cascante-Bonilla, Yu Hou, Yang Trista Cao, Hal Daumé, Rachel Rudinger

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV；VLM(comments)

Comments Project page: https://cece-vlm.github.io/

OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding

Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, Shuicheng Yan

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments NeurIPS-2024. Project page: https://lxtgh.github.io/project/omg_llava/

LLaVA-Gemma: Accelerating Multimodal Foundation Models with a Compact Language Model

Musashi Hinck, Matthew L. Olson, David Cobbley, Shao-Yen Tseng, Vasudev Lal

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.AI

Comments CVPR 2024, MMFM workshop. Authors 1 and 2 contributed equally. Models available at https://huggingface.co/intel/llava-gemma-2b/ and https://huggingface.co/intel/llava-gemma-7b/ Training code at https://github.com/IntelLabs/multimodal_cognitive_ai/tree/main/LLaVA-Gemma

VLMs for Videogame Data Annotation

用于视频游戏数据标注的视觉语言模型（VLMs）

Katrin Schmid, Iuri Frosio

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 该研究针对VLMs在视频游戏应用受限的问题，探究其用于游戏帧序列奖励信号标注的可行性，分析其在游戏问答中的不足并提出应对措施，还研究了输入参数对标注质量的影响。

RoboProcessBench: Benchmarking Process-Aware Understanding in Vision-Language Robotic Manipulation

RoboProcessBench：视觉语言机器人操作中的过程感知理解基准测试

Dayu Xia, Yue Shi, Yao Mu, Huiting Ji, Chaofan Ma, Yingjie Zhou, Hua Chen, Yang Liu, Jiezhang Cao, Guangtao Zhai

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Zhejiang University（浙江大学） ； Shanghai Jiao Tong University（上海交通大学） ； Tsinghua University（清华大学） ； China University of Mining Technology（中国矿业大学） 纠错

专题命中 VLM训练与架构 ：InternVL(abstract,abstract_cn)；vision-language model(abstract)；VLM(abstract_cn)

AI总结 提出RoboProcessBench基准，通过静态监控和动态推理两个维度、12个诊断问题家族，评估视觉语言模型在机器人操作中的过程感知理解能力，并基于58k问答对数据集验证了当前模型的局限性及后训练的有效性。

Element-Aware Group Learning for E-Commerce Image Generation

面向电商图像生成的元素感知组学习

Jingtong Chen, Jiahui Wang, Xue Zhao, ShaoGuo Liu, Minghao Li

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 针对电商图像生成中GRPO仅在完整提示层面分配信用的缺陷，提出EAGLE-GRPO，通过分解奖励并推导闭式解实现元素级信用分配，提升提示质量与生成图像性能。

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

MOON2.0：动态模态平衡多模态表示学习用于电商产品理解

Zhanheng Nie, Chenghan Fu, Daoze Zhang, Junxian Wu, Wanxian Guan, Pengjie Wang, Jian Xu, Bo Zheng

机构 * Alibaba Group（阿里巴巴集团） 纠错

专题命中 VLM训练与架构 ：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 MOON2.0通过动态模态平衡框架解决电商多模态数据中的模态不平衡、信息对齐不足和噪声处理问题，提出MoE、双层对齐和图像-文本共增强策略，提升零样本性能和多模态对齐质量。

Comments Accepted by the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 11 pages, 7 figures

FOCAL: Filtered On-device Continuous Activity Logging for Efficient Personal Desktop Summarization

FOCAL：过滤的本地连续活动日志用于高效的个人桌面摘要

Haoran Yin, Zhiyuan Wen, Jiannong Cao, Bo Yuan, Ruosong Yang

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；visual reasoning(abstract)

AI总结 FOCAL通过多代理系统实现高效的本地桌面流摘要，减少资源消耗并提升任务召回率。

Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

用于少样本工业视觉的小视觉语言模型的答案条件思维链蒸馏

Shubham Rao

机构 * Entropy AI Research Labs Private Limited（熵人工智能研究实验室私人有限公司） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 针对制造业视觉检测难题，提出答案条件思维链蒸馏法，利用最少标注数据让小型视觉语言模型适应新工业任务，经实验验证该方法在多任务中优于直接微调，提升了推理质量和模型性能。

Comments 11 pages, 5 figures, 8 tables

Heterogeneity-Adaptive Diffusion Schrodinger Bridge for PET-Guided Whole-Body MRI Translation

用于PET引导的全身MRI转换的异质性自适应扩散薛定谔桥

Chengbo Wang, Jiacheng Yu, Linjie Bian, Ming Qi, Xiaosheng Liu, Tongtong Che, Jichang Zhang, Shuyu Li, Shaoli Song, Xiuying Wang

机构 * The University of Sydney（悉尼大学） ； Fudan University Shanghai Cancer Center（复旦大学附属上海肿瘤医院） ； Beijing Normal University（北京师范大学） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 针对PET-MR扫描时间长的问题，提出异质性自适应扩散薛定谔桥（HA-DSB）框架，通过整合视觉语言模型嵌入及PET先验，利用双阶段引导机制，实现全身MRI转换，提升不同区域尤其是病变区域的转换质量。

Comments Accepted at MICCAI 2026