视觉大模型 / VLM

VLM-Eval: A General Evaluation on Video Large Language Models

Shuailin Li, Yuang Zhang, Yucheng Zhao, Qiuyue Wang, Fan Jia, Yingfei Liu, Tiancai Wang

专题命中 VLM训练与架构 ：VLM(title)；LLaVA(abstract)；分类 cs.CV

Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decoder

Abdelrahman Mohamed, Fakhraddin Alwajih, El Moatez Billah Nagoudi, Alcides Alcoba Inciarte, Muhammad Abdul-Mageed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted in ArabicNLP Conference

Improving Compositional Text-to-image Generation with Large Vision-Language Models

Song Wen, Guian Fang, Renrui Zhang, Peng Gao, Hao Dong, Dimitris Metaxas

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignment

Hongyu Hu, Tiancheng Lin, Jie Wang, Zhenbang Sun, Yi Xu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Distribution-Aware Prompt Tuning for Vision-Language Models

Eulrang Cho, Jooyeon Kim, Hyunwoo J. Kim

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to ICCV2023

Unsupervised Prototype Adapter for Vision-Language Models

Yi Zhang, Ce Zhang, Xueting Hu, Zhihai He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by PRCV 2023

Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models

Baoshuo Kan, Teng Wang, Wenpeng Lu, Xiantong Zhen, Weili Guan, Feng Zheng

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

SuS-X: Training-Free Name-Only Transfer of Vision-Language Models

Vishaal Udandarao, Ankush Gupta, Samuel Albanie

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at ICCV2023

ViGT: Proposal-free Video Grounding with Learnable Token in Transformer

Kun Li, Dan Guo, Meng Wang

专题命中 VLM训练与架构 ：grounding(title,abstract)；分类 cs.CV

Comments This paper has been accepted by SCIENCE CHINA Information Sciences

Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models

Kecheng Zheng, Wei Wu, Ruili Feng, Kai Zhu, Jiawei Liu, Deli Zhao, Zheng-Jun Zha, Wei Chen, Yujun Shen

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at ICCV 2023

X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks

Yan Zeng, Xinsong Zhang, Hang Li, Jiawei Wang, Jipeng Zhang, Wangchunshu Zhou

专题命中 VLM训练与架构 ：VLM(title,abstract)；分类 cs.CV

Comments 25 pages

Cross-Modal Concept Learning and Inference for Vision-Language Models

Yi Zhang, Ce Zhang, Yushun Tang, Zhihai He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

RSGPT: A Remote Sensing Vision Language Model and Benchmark

Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, Xiang Li

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV

Refined Vision-Language Modeling for Fine-grained Multi-modal Pre-training

Lisai Zhang, Qingcai Chen, Zhijian Chen, Yunpeng Han, Zhonghua Li, Zhao Cao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Work in progress, v0.2

CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model

Dingkang Liang, Jiahao Xie, Zhikang Zou, Xiaoqing Ye, Wei Xu, Xiang Bai

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2023

Task Residual for Tuning Vision-Language Models

Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, Xinchao Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2023

CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels

Siyuan Li, Li Sun, Qingli Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by AAAI23

Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding

Yuechen Wang, Wengang Zhou, Houqiang Li

专题命中 VLM训练与架构 ：grounding(title,abstract)；分类 cs.CV

Comments 11 pages, 4 figures, accepted by Findings of EMNLP 2021

Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models

Yue Zhang, Hongliang Fei, Dingcheng Li, Tan Yu, Ping Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Unsupervised Prompt Learning for Vision-Language Models

Tony Huang, Jack Chu, Fangyun Wei

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

V$^2$L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval

Wenhao Wang, Yifan Sun, Zongxin Yang, Yi Yang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR FGVC9 eBay eProduct Visual Search Challenge Rank 1st solution

A Simple Long-Tailed Recognition Baseline via Vision-Language Model

Teli Ma, Shijie Geng, Mengmeng Wang, Jing Shao, Jiasen Lu, Hongsheng Li, Peng Gao, Yu Qiao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts

Mu Cai, Haotian Liu, Dennis Park, Siva Karthik Mustikovela, Gregory P. Meyer, Yuning Chai, Yong Jae Lee

专题命中 VLM训练与架构 ：LLaVA(title,comments)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR2024. Project page: https://vip-llava.github.io/

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport

CoverPrune：基于最优传输的3D视觉语言模型的覆盖驱动型令牌剪枝

Peng Ling, Yingda Yin, Lingting Zhu, Weikai Chen, Shengju Qian, Zeyu Hu, Xin Wang, Wenming Yang

机构 * Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院） ； LIGHTSPEED 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 针对3D VLMs因大量视觉令牌导致的计算瓶颈，提出CoverPrune框架，将剪枝转化为最优传输问题，结合FST成本与SGS算法，实现高效剪枝且性能优异。

Comments Accepted to ECCV 2026 as an Oral Presentation

MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment

多模态代码切换：将视觉对象交织入语言以实现显式对象级对齐

Changhao Xiang, Shangyu Xing, Zhen Wu, Jianbing Zhang, Xinyu Dai

机构 * National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract,abstract_cn)；grounding(abstract)；分类 cs.CV、cs.LG

AI总结 针对现有多模态大语言模型的图像级对齐存在指称歧义的问题，提出多模态代码切换（MMCS）范式，构建含77.3万样本的数据集，仅用5万样本即可匹配或超越60万图像-文本对训练的模型，提升了视觉基础与感知能力。

RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs

RoRA：面向角色的区域分配，用于多模态大语言模型中的视觉令牌剪枝

Qiyanhui Lu, Han Wu, Rongjian Xu, Tingzhang Luo, Cheng Fan, Xinghao Chen, Minjing Dong, Jufeng Yang, Jianyuan Guo

专题命中 VLM训练与架构 ：LLaVA(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 本研究针对多模态大语言模型视觉令牌剪枝的高开销问题，提出无训练框架RoRA，通过角色导向的区域分配实现高效剪枝，在多个模型系列上优于基线，大幅降低推理时间且保留高准确率。

Comments 9 pages, 4 figures, 4 tables. Code is available at https://github.com/LukieLuu/RoRA

Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Prunin

学习在多模态大语言模型（MLLMs）中预测中间层注意力以进行视觉token剪枝

Yuyao Sun, Tao Deng, Shuang Li, Deqing Wang, Hao Geng, Minjun Yu

专题命中 VLM训练与架构 ：LLaVA(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 该研究针对MLLMs视觉token剪枝的固定层非最优及计算成本高的问题，提出MAP方法，实现仅保留5.56%视觉token时维持97.5%性能，获3.09倍端到端加速。

It's the Decoding Format, Not the Perturbation: Auditing Consistency-Based Selection for Vision-Language Test-Time Scaling

是解码格式，而非扰动：审计视觉语言模型测试时扩展的基于一致性的选择

Puzhuo Zheng, Hasan Kurban

专题命中 VLM训练与架构 ：LLaVA(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 该研究发现视觉语言模型测试时的选择效果由解码格式而非扰动决定，提出的扰动基选择（Pgs）在控制格式后无显著收益，说明其无法作为有效选择信号。

HarMoE: Multi-Source Chest Radiograph Pretraining with Dataset-Disentangled Experts

HarMoE：基于数据集解耦专家的多源胸部X射线预训练

Haozhe Luo, Ziyu Zhou, Shelley Zixin Shu, Mauricio Reyes

机构 * University of Bern（伯尔尼大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

AI总结 本研究提出HarMoE框架，通过解耦数据集专家从异构胸部X射线数据集学习，提升了放射学视觉-语言模型的零样本分类、分布外迁移等性能，相关代码与87.3万份数据集将公开。

ScaFE: Data-Efficient Scar Classification with LLM-Generated Clinical Feature Programs

ScaFE：基于大语言模型生成的临床特征程序实现数据高效的瘢痕分类

Ruman Wang, Hangting Ye

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.LG

AI总结 ScaFE将LLM临床知识转化为本地可执行的特征程序，结合轻量级随机森林，在瘢痕分类任务中实现数据高效、跨站点的高性能，且决策可审计，优于现有基线模型。