多模态大模型

Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens

Haohan Zheng, Zhenguo Zhang

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Cure or Poison? Embedding Instructions Visually Alters Hallucination in Vision-Language Models

Zhaochen Wang, Yiwei Wang, Yujun Cai

机构 * The University of Queensland（昆士兰大学） ； University of California, Merced（加州大学默塞德分校） 纠错

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Work in progress

Meta CLIP 2: A Worldwide Scaling Recipe

Yung-Sung Chuang, Yang Li, Dong Wang, Ching-Feng Yeh, Kehan Lyu, Ramya Raghavendra, James Glass, Lifei Huang, Jason Weston, Luke Zettlemoyer, Xinlei Chen, Zhuang Liu, Saining Xie, Wen-tau Yih, Shang-Wen Li, Hu Xu

机构 * FAIR, Meta（FAIR与Meta公司） ； MIT（麻省理工学院） ； Princeton University（普林斯顿大学） ； New York University（纽约大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments 10 pages

MR-CLIP: Efficient Metadata-Guided Learning of MRI Contrast Representations

Mehmet Yigit Avci, Pedro Borges, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge Cardoso

机构 * School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK（生物医学工程与成像科学学院，伦敦国王学院，伦敦，英国） ； deepc GMBH, Munich, Germany（deepc 德国慕尼黑分公司） 纠错

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Models

Xiaoyan Wang, Zeju Li, Yifan Xu, Jiaxing Qi, Zhifei Yang, Ruifei Ma, Xiangde Liu, Chao Zhang

机构 * Beijing Digital Native Digital City Research Center（北京数字原生数字城市研究中心） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICME2025

Skywork-R1V3 Technical Report

Wei Shen, Jiangbo Pei, Yi Peng, Xuchen Song, Yang Liu, Jian Peng, Haofeng Sun, Yunzhuo Hao, Peiyu Wang, Jianhao Zhang, Yahui Zhou

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

From Holistic to Localized: Local Enhanced Adapters for Efficient Visual Instruction Fine-Tuning

Pengkun Jiao, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang

机构 * Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University（上海智能信息处理关键实验室，复旦大学计算机学院） ； Shanghai Collaborative Innovation Center on Intelligent Visual Computing（上海智能视觉计算协同创新中心） ； Singapore Management University（新加坡管理大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2025

A Survey on Vision-Language-Action Models for Autonomous Driving

Sicong Jiang, Zilin Huang, Kangan Qian, Ziang Luo, Tianze Zhu, Yang Zhong, Yihong Tang, Menglin Kong, Yunlong Wang, Siwen Jiao, Hao Ye, Zihao Sheng, Xin Zhao, Tuopu Wen, Zheng Fu, Sikai Chen, Kun Jiang, Diange Yang, Seongjin Choi, Lijun Sun

机构 * McGill University（麦吉尔大学） ； Tsinghua University（清华大学） ； Xiaomi Corporation（小米公司） ； University of Wisconsin–Madison（威斯康星大学麦迪逊分校） ； University of Minnesota–Twin Cities（明尼苏达大学双城分校） ； State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（智能绿色车辆与移动国家重点实验室，清华大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Generalizing vision-language models to novel domains: A comprehensive survey

Xinyao Li, Jingjing Li, Fengling Li, Lei Zhu, Yang Yang, Heng Tao Shen

机构 * University of Electronic Science and Technology of China（电子科技大学） ； University of Technology Sydney（技术学院） ； Tongji University（同济大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Unifying Biomedical Vision-Language Expertise: Towards a Generalist Foundation Model via Multi-CLIP Knowledge Distillation

Shansong Wang, Zhecheng Jin, Mingzhe Hu, Mojtaba Safari, Feng Zhao, Chih-Wei Chang, Richard LJ Qiu, Justin Roper, David S. Yu, Xiaofeng Yang

机构 * Department of Radiation Oncology, Winship Cancer Institute, Emory University School of Medicine（放射肿瘤科，Winship癌症研究所，埃默里大学医学院） ； Department of Biomedical Engineering, College of Engineering, Georgia Institute of Technology（生物医学工程系，工程学院，佐治亚理工学院） ； Department of Computer Science and Mathematics, Laney Graduate School, Emory University（计算机科学与数学系，兰尼研究生院，埃默里大学） ； School of Electrical and Computer Engineering, College of Engineering, Georgia Institute of Technology（电气与计算机工程系，工程学院，佐治亚理工学院） 纠错

专题命中 图文多模态 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets

Kaiyuan Chen, Shuangyu Xie, Zehan Ma, Pannag R Sanketi, Ken Goldberg

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CL、cs.AI

SensorLM: Learning the Language of Wearable Sensors

Yuwei Zhang, Kumar Ayush, Siyuan Qiao, A. Ali Heydari, Girish Narayanswamy, Maxwell A. Xu, Ahmed A. Metwally, Shawn Xu, Jake Garrison, Xuhai Xu, Tim Althoff, Yun Liu, Pushmeet Kohli, Jiening Zhan, Mark Malhotra, Shwetak Patel, Cecilia Mascolo, Xin Liu, Daniel McDuff, Yuzhe Yang

机构 * Google DeepMind（谷歌DeepMind） ； Google Research（谷歌研究） ； University of Cambridge（剑桥大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks

Mengzhao Jia, Wenhao Yu, Kaixin Ma, Tianqing Fang, Zhihan Zhang, Siru Ouyang, Hongming Zhang, Dong Yu, Meng Jiang

机构 * University of Notre Dame（诺丁汉大学） ； Tencent AI Seattle Lab（腾讯AI西雅图实验室） ； UIUC（伊利诺伊大学香槟分校） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Our code is available at https://github.com/tencent-ailab/Leopard

Curriculum Learning with Quality-Driven Data Selection

Biao Wu, Ling Chen

机构 * Australian Artificial Intelligence Institute（澳大利亚人工智能研究所） 纠错

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CL、cs.AI

CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling

Jihai Zhang, Xiaoye Qu, Tong Zhu, Yu Cheng

机构 * First Author Affiliation（第一作者机构） ； Second Author Affiliation（第二作者机构） 纠错

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

FG-CLIP: Fine-Grained Visual and Textual Alignment

Chunyu Xie, Bin Wang, Fanjing Kong, Jincheng Li, Dawei Liang, Gengshen Zhang, Dawei Leng, Yuhui Yin

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICML 2025

Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training

Xinsong Zhang, Yarong Zeng, Xinting Huang, Hu Hu, Runquan Xie, Han Hu, Zhanhui Kang

机构 * Tencent（腾讯） 纠错

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction

Yuanchang Ye, Weiyan Wen

专题命中 图文多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICIPCA 2025

PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks

Jianyu Wu, Hao Yang, Xinhua Zeng, Guibing He, Zhiyu Chen, Zihui Li, Xiaochuan Zhang, Yangyang Ma, Run Fang, Yang Liu

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Post-pre-training for Modality Alignment in Vision-Language Foundation Models

Shin'ya Yamaguchi, Dewei Feng, Sekitoshi Kanai, Kazuki Adachi, Daiki Chijiwa

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2025; Code: https://github.com/yshinya6/clip-refine

CameraBench: Benchmarking Visual Reasoning in MLLMs via Photography

I-Sheng Fang, Jun-Cheng Chen

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

ReferGPT: Towards Zero-Shot Referring Multi-Object Tracking

Tzoulio Chamiti, Leandro Di Bella, Adrian Munteanu, Nikos Deligiannis

专题命中 图文多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted CVPR 2025 Workshop on Distillation of Foundation Models for Autonomous Driving

BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature

Alejandro Lozano, Min Woo Sun, James Burgess, Liangyu Chen, Jeffrey J Nirschl, Jeffrey Gu, Ivan Lopez, Josiah Aklilu, Austin Wolfgang Katzer, Collin Chiu, Anita Rau, Xiaohan Wang, Yuhui Zhang, Alfred Seunghoon Song, Robert Tibshirani, Serena Yeung-Levy

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference

Hao Yin, Guangzong Si, Zilei Wang

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Treble Counterfactual VLMs: A Causal Approach to Hallucination

Shawn Li, Jiashu Qu, Yuxiao Zhou, Yuehan Qin, Tiankai Yang, Yue Zhao

专题命中 图文多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Models

Zilu Guo, Hongbin Lin, Zhihao Yuan, Chaoda Zheng, Pengshuo Qiu, Dongzhi Jiang, Renrui Zhang, Chun-Mei Feng, Zhen Li

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Technical Report

DiffCLIP: Differential Attention Meets CLIP

Hasan Abed Al Kader Hammoud, Bernard Ghanem

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Under review

From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Models

Muzhi Dai, Jiashuo Sun, Zhiyuan Zhao, Shixuan Liu, Rui Li, Junyu Gao, Xuelong Li

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency

Che Liu, Cheng Ouyang, Yinda Chen, Cesar César Quilodrán-Casas, Lei Ma, Jie Fu, Yike Guo, Anand Shah, Wenjia Bai, Rossella Arcucci

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

NanoVLMs: How small can we go and still make coherent Vision Language Models?

Mukund Agarwalla, Himanshu Kumar, Raj Dandekar, Rajat Dandekar, Sreedath Panat

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 8 figures, 3 tables