视觉大模型 / VLM

JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration

Yunlong Lin, Zixu Lin, Haoyu Chen, Panwang Pan, Chenxin Li, Sixiang Chen, Yeying Jin, Wenbo Li, Xinghao Ding

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV

Comments 25 pages, 15 figures

Window Token Concatenation for Efficient Visual Large Language Models

Yifan Li, Wentao Bao, Botao Ye, Zhen Tan, Tianlong Chen, Huan Liu, Yu Kong

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV

BioX-CPath: Biologically-driven Explainable Diagnostics for Multistain IHC Computational Pathology

Amaya Gallagher-Syed, Henry Senior, Omnia Alwazzan, Elena Pontarini, Michele Bombardieri, Costantino Pitzalis, Myles J. Lewis, Michael R. Barnes, Luca Rossi, Gregory Slabaugh

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV

Comments Accepted for publication at CVPR 2025

CLIP-SLA: Parameter-Efficient CLIP Adaptation for Continuous Sign Language Recognition

Sarah Alyami, Hamzah Luqman

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation

Wei Chen, Lin Li, Yongqi Yang, Bin Wen, Fan Yang, Tingting Gao, Yu Wu, Long Chen

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments 22 pages, Accepted by CVPR 2025

Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation

Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, Fei Wang, Yujia Xie, Mahmoud Khademi, Ziyi Yang, Hany Awadalla, Julia Gong, Houdong Hu, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Yu Gu, Cliff Wong, Mu Wei, Tristan Naumann, Muhao Chen, Matthew P. Lungren, Akshay Chaudhari, Serena Yeung-Levy, Curtis P. Langlotz, Sheng Wang, Hoifung Poon

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV

Journal ref Nature Communications volume 16, Article number: 3108 (2025)

DetailCLIP: Detail-Oriented CLIP for Fine-Grained Tasks

Amin Karimi Monsefi, Kishore Prakash Sailaja, Ali Alilooee, Ser-Nam Lim, Rajiv Ramnath

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted in SSI-FM Workshop of ICLR 2025

Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-tasks

Jonathan Salfity, Selma Wanna, Minkyu Choi, Mitch Pryor

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.LG

Comments 8 pages, 3 figures. IROS 2024 Submission

Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data

Haoxin Li, Boyang Li

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Can language-guided unsupervised adaptation improve medical image classification using unpaired images and texts?

Umaima Rahman, Raza Imam, Mohammad Yaqub, Boulbaba Ben Amor, Dwarikanath Mahapatra

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Conference paper at International Symposium on Biomedical Imaging (ISBI) 2025

TULIP: Token-length Upgraded CLIP

Ivona Najdenkoska, Mohammad Mahdi Derakhshani, Yuki M. Asano, Nanne van Noord, Marcel Worring, Cees G. M. Snoek

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Video-Panda: Parameter-efficient Alignment for Encoder-free Video-Language Models

Jinhui Yi, Syed Talal Wasim, Yanan Luo, Muzammal Naseer, Juergen Gall

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV

Comments CVPR 2025 camera-ready version

Efficient Multi-Instance Generation with Janus-Pro-Dirven Prompt Parsing

Fan Qi, Yu Duan, Changsheng Xu

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV

VTD-CLIP: Video-to-Text Discretization via Prompting CLIP

Wencheng Zhu, Yuexin Wang, Hongxuan Li, Pengfei Zhu, Qinghua Hu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

GOAL: Global-local Object Alignment Learning

Hyungyu Choi, Young Kyun Jang, Chanho Eom

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments 16 pages, 5 figures

Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions

Dong Jing, Nanyi Fei, Zhiwu Lu

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI

Visual Persona: Foundation Model for Full-Body Human Customization

Jisu Nam, Soowon Son, Zhan Xu, Jing Shi, Difan Liu, Feng Liu, Aashish Misraa, Seungryong Kim, Yang Zhou

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments CVPR 2025, Project page is available at https://cvlab-kaist.github.io/Visual-Persona

Aligning Multimodal LLM with Human Preference: A Survey

Tao Yu, Yi-Fan Zhang, Chaoyou Fu, Junkang Wu, Jinda Lu, Kun Wang, Xingyu Lu, Yunhang Shen, Guibin Zhang, Dingjie Song, Yibo Yan, Tianlong Xu, Qingsong Wen, Zhang Zhang, Yan Huang, Liang Wang, Tieniu Tan

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment

Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks

Miran Heo, Min-Hung Chen, De-An Huang, Sifei Liu, Subhashree Radhakrishnan, Seon Joo Kim, Yu-Chiang Frank Wang, Ryo Hachiuma

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments CVPR 2025, Project page: https://miranheo.github.io/omni-rgpt/

Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignment

Mayug Maniparambil, Raiymbek Akshulakov, Yasser Abdelaziz Dahou Djilali, Sanath Narayan, Ankit Singh, Noel E. O'Connor

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted CVPR 2025; First two authors contributed equally;

ExCap3D: Expressive 3D Scene Understanding via Object Captioning with Varying Detail

Chandan Yeshwanth, David Rozenberszki, Angela Dai

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV

Comments Project page: https://cy94.github.io/excap3d/, Video: https://www.youtube.com/watch?v=SQRV1l_0oY0

When Less is Enough: Adaptive Token Reduction for Efficient Image Representation

Eduard Allakhverdov, Elizaveta Goncharova, Andrey Kuznetsov

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV

Comments 10 pages, 8 figures

VASparse: Towards Efficient Visual Hallucination Mitigation via Visual-Aware Token Sparsification

Xianwei Zhuang, Zhihong Zhu, Yuxin Xie, Liming Liang, Yuexian Zou

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

Towards Automatic Evaluation for Image Transcreation

Simran Khanuja, Vivek Iyer, Claire He, Graham Neubig

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV

Comments To be presented at NAACL 2025

Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference

Siyuan Wang, Dianyi Wang, Chengxing Zhou, Zejun Li, Zhihao Fan, Xuanjing Huang, Zhongyu Wei

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Morphing Tokens Draw Strong Masked Image Models

Taekyung Kim, Byeongho Heo, Dongyoon Han

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments 24 pages, 16 tables, 8 figures. To be presented at ICLR'25

UniCrossAdapter: Multimodal Adaptation of CLIP for Radiology Report Generation

Yaxiong Chen, Chuang Du, Chunlei Li, Jingliang Hu, Yilei Shi, Shengwu Xiong, Xiao Xiang Zhu, Lichao Mou

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments MICCAI 2024 Workshop

EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions

Kai Chen, Yunhao Gou, Runhui Huang, Zhili Liu, Daxin Tan, Jing Xu, Chunwei Wang, Yi Zhu, Yihan Zeng, Kuo Yang, Dingdong Wang, Kun Xiang, Haoyuan Li, Haoli Bai, Jianhua Han, Xiaohui Li, Weike Jin, Nian Xie, Yu Zhang, James T. Kwok, Hengshuang Zhao, Xiaodan Liang, Dit-Yan Yeung, Xiao Chen, Zhenguo Li, Wei Zhang, Qun Liu, Jun Yao, Lanqing Hong, Lu Hou, Hang Xu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025. Project Page: https://emova-ollm.github.io/

Vision-Speech Models: Teaching Speech Models to Converse about Images

Amélie Royer, Moritz Böhle, Gabriel de Marmiesse, Laurent Mazaré, Neil Zeghidour, Alexandre Défossez, Patrick Pérez

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Efficient Transfer Learning for Video-language Foundation Models

Haoxing Chen, Zizheng Huang, Yan Hong, Yanshuo Wang, Zhongcai Lyu, Zhuoer Xu, Jun Lan, Zhangxuan Gu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025