视觉大模型 / VLM

LADA: Scalable Label-Specific CLIP Adapter for Continual Learning

Mao-Lin Luo, Zi-Hao Zhou, Tong Wei, Min-Ling Zhang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at ICML 2025

ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG

Zilun Zhang, Haozhan Shen, Tiancheng Zhao, Zian Guan, Bin Chen, Yuhao Wang, Xu Jia, Yuxiang Cai, Yongheng Shang, Jianwei Yin

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； Binjiang Research Institute of Zhejiang University（浙江大学滨江研究院） ； School of Software Engineering of Zhejiang University（浙江大学软件工程学院） ； Polytechnic Institute of Zhejiang University（浙江大学 polytechnic 院） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Geoscience and Remote Sensing Magazine

AdaTP: Attention-Debiased Token Pruning for Video Large Language Models

Fengyuan Sun, Leqi Shen, Hui Chen, Sicheng Zhao, Jungong Han, Guiguang Ding

机构 * School of Software, Tsinghua University（清华大学软件学院） ； BNRist, Tsinghua University（清华大学BNRist） ； Department of Automation, Tsinghua University（清华大学自动化系） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)

Subba Reddy Oota, Akshett Jindal, Ishani Mondal, Khushbu Pahwa, Satya Sai Srinath Namburi, Manish Shrivastava, Maneesh Singh, Bapi S. Raju, Manish Gupta

机构 * Technische Universität Berlin（柏林技术大学） ； IIIT Hyderabad（海得拉巴国家理工学院） ； Univ of Maryland（马里兰大学） ； Rice Univ（Rice 大学） ； Univ of Wisconsin - Madison（威斯康星大学麦迪逊分校） ； Spector Inc（Spector 公司） ； Microsoft（微软公司） 纠错

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.AI、cs.LG

Comments 30 pages, 22 figures, The Thirteenth International Conference on Learning Representations, ICLR-2025, Singapore. https://openreview.net/pdf?id=xkgfLXZ4e0

Journal ref ICLR-2025, Singapore

Rethinking Causal Mask Attention for Vision-Language Inference

Xiaohuan Pei, Tao Huang, YanXiang Ma, Chang Xu

机构 * School of Computer Science, The University of Sydney（悉尼大学计算机科学学院） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach

Changdae Oh, Zhen Fang, Shawn Im, Xuefeng Du, Yixuan Li

机构 * Changdae Oh ； Zhen Fang ； Shawn Im ； Xuefeng Du ； Yixuan Li 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments ICML 2025 camera-ready

TNG-CLIP:Training-Time Negation Data Generation for Negation Awareness of CLIP

Yuliang Cai, Jesse Thomason, Mohammad Rostami

机构 * University of Southern California（南加州大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 3 figures

AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training

Huishuai Zhang, Bohan Wang, Luoxin Chen

机构 * Wangxuan Institute of Computer Technology, Peking University（北京大学王宣计算机技术研究所） ； University of Science and Technology of China（中国科学技术大学） ； State Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室） 纠错

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.AI、cs.LG

MedBLIP: Fine-tuning BLIP for Medical Image Captioning

Manshi Limbu, Diwita Banerjee

机构 * George Mason University（乔治·马歇尔大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Visual Agentic Reinforcement Fine-Tuning

Ziyu Liu, Yuhang Zang, Yushan Zou, Zijian Liang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, Jiaqi Wang

机构 * Shanghai Jiaotong University（上海交通大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； The Chinese University of Hong Kong（香港中文大学） ； Wuhan University（武汉大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments project url: https://github.com/Liuziyu77/Visual-RFT/tree/main/Visual-ARFT

EfficientLLM: Efficiency in Large Language Models

Zhengqing Yuan, Weixiang Sun, Yixin Liu, Huichi Zhou, Rong Zhou, Yiyang Li, Zheyuan Zhang, Wei Song, Yue Huang, Haolong Jia, Keerthiram Murugesan, Yu Wang, Lifang He, Jianfeng Gao, Lichao Sun, Yanfang Ye

机构 * University of Notre Dame（诺丁汉大学） ； Lehigh University（莱斯大学） ； Imperial College London（伦敦帝国理工学院） ； Rutgers University（罗格斯大学） ； International Business Machines Corporation (IBM)（国际商业机器公司（IBM）） ； University of Illinois Chicago（伊利诺伊大学芝加哥分校） ； Microsoft Research（微软研究院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Evaluating the Efficacy of Prompt-Engineered Large Multimodal Models Versus Fine-Tuned Vision Transformers in Image-Based Security Applications

Fouad Trad, Ali Chehab

机构 * Electrical and Computer Engineering（电气与计算机工程） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Journal ref Published in ACM Transactions on Intelligent Systems and Technology, 2025

CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback

Yixin Wan, Kai-Wei Chang

机构 * Department of Computer Science University of California, Los Angeles（计算机科学系，加州大学洛杉矶分校） 纠错

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.AI

In-Context Learning for Label-Efficient Cancer Image Classification in Oncology

Mobina Shrestha, Bishwas Mandal, Vishal Mandal, Asis Shrestha

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Seed1.5-VL Technical Report

Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, Jingji Chen, Jingjia Huang, Kang Lei, Liping Yuan, Lishu Luo, Pengfei Liu, Qinghao Ye, Rui Qian, Shen Yan, Shixiong Zhao, Shuai Peng, Shuangye Li, Sihang Yuan, Sijin Wu, Tianheng Cheng, Weiwei Liu, Wenqian Wang, Xianhan Zeng, Xiao Liu, Xiaobo Qin, Xiaohan Ding, Xiaojun Xiao, Xiaoying Zhang, Xuanwei Zhang, Xuehan Xiong, Yanghua Peng, Yangrui Chen, Yanwei Li, Yanxu Hu, Yi Lin, Yiyuan Hu, Yiyuan Zhang, Youbin Wu, Yu Li, Yudong Liu, Yue Ling, Yujia Qin, Zanbo Wang, Zhiwu He, Aoxue Zhang, Bairen Yi, Bencheng Liao, Can Huang, Can Zhang, Chaorui Deng, Chaoyi Deng, Cheng Lin, Cheng Yuan, Chenggang Li, Chenhui Gou, Chenwei Lou, Chengzhi Wei, Chundian Liu, Chunyuan Li, Deyao Zhu, Donghong Zhong, Feng Li, Feng Zhang, Gang Wu, Guodong Li, Guohong Xiao, Haibin Lin, Haihua Yang, Haoming Wang, Heng Ji, Hongxiang Hao, Hui Shen, Huixia Li, Jiahao Li, Jialong Wu, Jianhua Zhu, Jianpeng Jiao, Jiashi Feng, Jiaze Chen, Jianhui Duan, Jihao Liu, Jin Zeng, Jingqun Tang, Jingyu Sun, Joya Chen, Jun Long, Junda Feng, Junfeng Zhan, Junjie Fang, Junting Lu, Kai Hua, Kai Liu, Kai Shen, Kaiyuan Zhang, Ke Shen, Ke Wang, Keyu Pan, Kun Zhang, Kunchang Li, Lanxin Li, Lei Li, Lei Shi, Li Han, Liang Xiang, Liangqiang Chen, Lin Chen, Lin Li, Lin Yan, Liying Chi, Longxiang Liu, Mengfei Du, Mingxuan Wang, Ningxin Pan, Peibin Chen, Pengfei Chen, Pengfei Wu, Qingqing Yuan, Qingyao Shuai, Qiuyan Tao, Renjie Zheng, Renrui Zhang, Ru Zhang, Rui Wang, Rui Yang, Rui Zhao, Shaoqiang Xu, Shihao Liang, Shipeng Yan, Shu Zhong, Shuaishuai Cao, Shuangzhi Wu, Shufan Liu, Shuhan Chang, Songhua Cai, Tenglong Ao, Tianhao Yang, Tingting Zhang, Wanjun Zhong, Wei Jia, Wei Weng, Weihao Yu, Wenhao Huang, Wenjia Zhu, Wenli Yang, Wenzhi Wang, Xiang Long, XiangRui Yin, Xiao Li, Xiaolei Zhu, Xiaoying Jia, Xijin Zhang, Xin Liu, Xinchen Zhang, Xinyu Yang, Xiongcai Luo, Xiuli Chen, Xuantong Zhong, Xuefeng Xiao, Xujing Li, Yan Wu, Yawei Wen, Yifan Du, Yihao Zhang, Yining Ye, Yonghui Wu, Yu Liu, Yu Yue, Yufeng Zhou, Yufeng Yuan, Yuhang Xu, Yuhong Yang, Yun Zhang, Yunhao Fang, Yuntao Li, Yurui Ren, Yuwen Xiong, Zehua Hong, Zehua Wang, Zewei Sun, Zeyu Wang, Zhao Cai, Zhaoyue Zha, Zhecheng An, Zhehui Zhao, Zhengzhuo Xu, Zhipeng Chen, Zhiyong Wu, Zhuofan Zheng, Zihao Wang, Zilong Huang, Ziyu Zhu, Zuquan Song

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

VladVA: Discriminative Fine-tuning of LVLMs

Yassine Ouali, Adrian Bulat, Alexandros Xenos, Anestis Zaganidis, Ioannis Maniadis Metaxas, Brais Martinez, Georgios Tzimiropoulos

机构 * Samsung AI Cambridge（三星AI剑桥） ； Technical University of Iasi（伊阿西技术大学） ； Queen Mary University of London（伦敦女王学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Published at CVPR 2025

Image-Text Relation Prediction for Multilingual Tweets

Matīss Rikters, Edison Marrese-Taylor

机构 * Artificial Intelligence Research Center（人工智能研究中心） ； National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Journal ref Published in Proceedings of the 1st Workshop on Nordic-Baltic Responsible Evaluation and Alignment of Language, NoDaLiDa - Baltic HLT 2025

SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning

Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, Sam Kwong

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Token Sequence Compression for Efficient Multimodal Computing

Yasmine Omri, Parth Shroff, Thierry Tambe

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.AI

$\texttt{Complex-Edit}$: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmark

Siwei Yang, Mude Hui, Bingchen Zhao, Yuyin Zhou, Nataniel Ruiz, Cihang Xie

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://ucsc-vlaa.github.io/Complex-Edit/, Dataset: https://huggingface.co/datasets/UCSC-VLAA/Complex-Edit

IterIS: Iterative Inference-Solving Alignment for LoRA Merging

Hongxu Chen, Runshi Li, Bowei Zhu, Zhen Wang, Long Chen

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.AI、cs.LG

ResiDual Transformer Alignment with Spectral Decomposition

Lorenzo Basile, Valentino Maiorca, Luca Bortolussi, Emanuele Rodolà, Francesco Locatello

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Published in Transactions on Machine Learning Research (TMLR)

FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations

Cheng-Yu Hsieh, Pavan Kumar Anasosalu Vasu, Fartash Faghri, Raviteja Vemulapalli, Chun-Liang Li, Ranjay Krishna, Oncel Tuzel, Hadi Pouransari

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.LG

VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Models

Dahun Kim, AJ Piergiovanni, Ganesh Mallya, Anelia Angelova

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025, project page at https://github.com/google-deepmind/video_comp

URECA: Unique Region Caption Anything

Sangbeom Lim, Junwan Kim, Heeji Yoon, Jaewoo Jung, Seungryong Kim

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://cvlab-kaist.github.io/URECA Code: https://github.com/cvlab-kaist/URECA

Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

Erfan Shayegani, G M Shahariar, Sara Abdali, Lei Yu, Nael Abu-Ghazaleh, Yue Dong

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.AI、cs.LG

RoboAct-CLIP: Video-Driven Pre-training of Atomic Action Understanding for Robotics

Zhiyuan Zhang, Yuxin He, Yong Sun, Junyu Shi, Lijiang Liu, Qiang Nie

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.AI、cs.LG

Comments IROS 2025

FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputs

Mothilal Asokan, Kebin Wu, Fatima Albreiki

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features

Jewon Lee, Ki-Ung Song, Seungmin Yang, Donguk Lim, Jaeyeon Kim, Wooksu Shin, Bo-Kyeong Kim, Yong Jae Lee, Tae-Ho Kim

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments accepted at CVPR 2025 Workshop on ELVM

Enhancing Domain Adaptation through Prompt Gradient Alignment

Hoang Phan, Lam Tran, Quyen Tran, Trung Le

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2024