多模态大模型

Adaptive Forensic Feature Refinement via Intrinsic Importance Perception

通过内在重要性感知实现自适应的 forensic 特征细化

Jiazhen Yang, Junjun Zheng, Kejia Chen, Xiangheng Kong, Jie Lei, Zunlei Feng, Bingde Hu, Yang Gao

机构 * Zhejiang University（浙江大学） ； Alibaba Inc（阿里巴巴公司） ； Zhejiang University of Technology（浙江工业大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

AI总结 本文提出I2P框架，通过自适应识别关键层表示并约束任务驱动的参数更新，提升SID任务的特定性同时保留预训练表示的可迁移性。

VOLMO: Versatile and Open Large Models for Ophthalmology

VOLMO：面向眼科学的多功能和开放型大模型

Zhenyue Qin, Younjoon Chung, Elijah Lee, Wanyue Feng, Xuguang Ai, Serina Applebaum, Minjie Zou, Yang Liu, Pan Xiao, Mac Singer, Amisha Dave, Aidan Gilson, Tiarnan D. L. Keenan, Emily Y. Chew, Zhiyong Lu, Yih-Chung Tham, Ron Adelman, Luciano V. Del Priore, Qingyu Chen

机构 * Department of Biomedical Informatics & Data Science, Yale University（耶鲁大学生物医学信息学与数据科学系） ； Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University（卡内基梅隆大学雷和斯蒂芬妮·兰德计算生物学系） ； Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨洛林医学院） ； Department of Radiology, Washington University in Saint Louis（圣路易斯华盛顿大学放射科） ； National Eye Institute, National Institutes of Health（国家卫生研究院眼科研究所） ； National Library of Medicine, National Institutes of Health（国家卫生研究院国家医学图书馆） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

AI总结 VOLMO提出了一种通用框架，用于开发专门的眼科多模态大语言模型，通过预训练、微调和临床推理三个阶段，提升了眼科疾病筛查和诊断的性能。

HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoning

HoloLLM：面向语言基础的人感知与推理的多感官基础模型

Chuhao Zhou, Jianfei Yang

机构 * MARS Lab, Nanyang Technological University（南洋理工大学MARS实验室） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 HoloLLM通过整合LiDAR、红外、毫米波雷达和Wi-Fi等多感官数据，提升语言基础的人感知与推理能力，实验表明其在真实场景中的性能提升达30%。

Comments Camera-ready version. Accepted at NeurIPS 2025

E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis

E²-LLM：连接神经信号与可解释的情感分析

Fei Ma, Han Lin, Yifan Xie, Hongwei Ren, Xiaoyu Shen, Wenbo Ding, Qi Tian

机构 * Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室） ； Zhejiang University（浙江大学） ； Tsinghua University（清华大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Eastern Institute of Technology（东方技术研究所） ； Huawei（华为） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.AI

AI总结 E²-LLM通过整合预训练EEG编码器与Qwen-based LLM，实现了可解释的情绪分析，展示了模型扩展在情感识别和可解释性上的优势。

Comments 11 pages

Explore How to Inject Beneficial Noise in MLLMs

Ruishu Zhu, Sida Huang, Ziheng Jiao, Hongyuan Zhang

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by AAAI 2026

Reimagining Safety Alignment with An Image

Yifan Xia, Guorui Chen, Wenqian Yu, Zhijiang Li, Philip Torr, Jindong Gu

机构 * School of Information Management, Wuhan University, Wuhan, China（武汉大学信息管理学院） ； Torr Vision Group, University of Oxford, Oxford, United Kingdom（牛津大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.AI

MoralCLIP: Contrastive Alignment of Vision-and-Language Representations with Moral Foundations Theory

Ana Carolina Condez, Diogo Tavares, João Magalhães

机构 * NOVA LINCS, NOVA School of Science and Technology（NOVA LINCS，NOVA科学与技术学院） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Updated version: corresponds to the ACM MM '25 published paper and includes full appendix material

Law of Vision Representation in MLLMs

Shijia Yang, Bohan Zhai, Quanzeng You, Jianbo Yuan, Hongxia Yang, Chenfeng Xu

机构 * Stanford University（斯坦福大学） ； UC Berkeley（加州大学伯克利分校） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.CV

Comments The code is available at https://github.com/bronyayang/Law_of_Vision_Representation_in_MLLMs

Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback

Yang Chen, Yufan Shen, Wenxuan Huang, Sheng Zhou, Qunshu Lin, Xinyu Cai, Zhi Yu, Jiajun Bu, Botian Shi, Yu Qiao

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

FedVLMBench: Benchmarking Federated Fine-Tuning of Vision-Language Models

Weiying Zheng, Ziyue Lin, Pengxin Guo, Yuyin Zhou, Feifei Wang, Liangqiong Qu

机构 * The University of Hong Kong（香港大学） ； UC Santa Cruz（圣克拉拉大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；multimodal foundation model(abstract)；分类 cs.CV

Mentor3AD: Feature Reconstruction-based 3D Anomaly Detection via Multi-modality Mentor Learning

Hanzhe Liang

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments arXiv admin comment: This version has been removed by arXiv administrators as the submitter did not have the rights to agree to the license at the time of submission

MLLMs are Deeply Affected by Modality Bias

Xu Zheng, Chenfei Liao, Yuqian Fu, Kaiyu Lei, Yuanhuiyi Lyu, Lutao Jiang, Bin Ren, Jialei Chen, Jiawen Wang, Chengxin Li, Linfeng Zhang, Danda Pani Paudel, Xuanjing Huang, Yu-Gang Jiang, Nicu Sebe, Dacheng Tao, Luc Van Gool, Xuming Hu

机构 * HKUST(GZ)（香港科技大学（广州）） ； CSE, HKUST（香港科技大学计算机科学与工程系） ； Xi’an Jiaotong University（西安交通大学） ； University of Pisa, IT（比萨大学） ； University of Trento, IT（特伦特大学） ； Nagoya University（名古屋大学） ； China University of Mining & Technology, Beijing（中国矿业大学（北京）） ； Tongji University（同济大学） ； SPIC Energy Science and Technology Research Institute（SPIC能源科学与技术研究院） ； Shanghai Jiao Tong University（上海交通大学） ； Fudan University（复旦大学） ； College of Computing & Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院） 纠错

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.AI

A Survey on Remote Sensing Foundation Models: From Vision to Multimodality

Ziyue Huang, Hongxi Yan, Qiqi Zhan, Shuai Yang, Mingming Zhang, Chenkai Zhang, YiMing Lei, Zeming Liu, Qingjie Liu, Yunhong Wang

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；multimodal foundation model(abstract)；分类 cs.CV

Optimal Transport Adapter Tuning for Bridging Modality Gaps in Few-Shot Remote Sensing Scene Classification

Zhong Ji, Ci Liu, Jingren Liu, Chen Tang, Yanwei Pang, Xuelong Li

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Towards Improved Text-Aligned Codebook Learning: Multi-Hierarchical Codebook-Text Alignment with Long Text

Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Junteng Zhao, Yunming Ye, Kola Ye, Yao He

专题命中 多模态训练与对齐 ：multi-modal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

Aligning Large Language Models and Geometric Deep Models for Protein Representation

Dong Shu, Bingbing Duan, Kai Guo, Kaixiong Zhou, Jiliang Tang, Mengnan Du

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.AI

Comments 37 pages, 10 figures

UniGS: Unified Language-Image-3D Pretraining with Gaussian Splatting

Haoyuan Li, Yanpeng Zhou, Tao Tang, Jifei Song, Yihan Zeng, Michael Kampffmeyer, Hang Xu, Xiaodan Liang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ICLR 2025; Corrected citation of Uni3D;

VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset

Jing Liu, Sihan Chen, Xingjian He, Longteng Guo, Xinxin Zhu, Weining Wang, Jinhui Tang

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Preprint version w/o audio files embeded in PDF. Audio embeded version can be found on project page or github

BSAFusion: A Bidirectional Stepwise Feature Alignment Network for Unaligned Medical Image Fusion

Huafeng Li, Dayong Su, Qing Cai, Yafei Zhang

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI2025

SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding

Hao Li, Changyao Tian, Jie Shao, Xizhou Zhu, Zhaokai Wang, Jinguo Zhu, Wenhan Dou, Xiaogang Wang, Hongsheng Li, Lewei Lu, Jifeng Dai

专题命中 多模态训练与对齐 ：multimodal(abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

SafaRi:Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentation

Sayan Nag, Koustava Goswami, Srikrishna Karanam

专题命中 多模态训练与对齐 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at ECCV 2024

MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning

Xiangyu Zhao, Xiangtai Li, Haodong Duan, Haian Huang, Yining Li, Kai Chen, Hua Yang

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Emu: Generative Pretraining in Multimodality

Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, Xinlong Wang

专题命中 多模态训练与对齐 ：multimodal(abstract)；image-text(abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments Accepted to ICLR 2024. Code and Models: https://github.com/baaivision/Emu

VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models

Ziyi Yin, Muchao Ye, Tianrong Zhang, Jiaqi Wang, Han Liu, Jinghui Chen, Ting Wang, Fenglong Ma

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments AAAI 2024, 11 pages

ImageBind-LLM: Multi-modality Instruction Tuning

Jiaming Han, Renrui Zhang, Wenqi Shao, Peng Gao, Peng Xu, Han Xiao, Kaipeng Zhang, Chris Liu, Song Wen, Ziyu Guo, Xudong Lu, Shuai Ren, Yafei Wen, Xiaoxin Chen, Xiangyu Yue, Hongsheng Li, Yu Qiao

专题命中 多模态训练与对齐 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Code is available at https://github.com/OpenGVLab/LLaMA-Adapter

NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection

Yu Chen, Yang Yu, Rongrong Ni, Yao Zhao, Haoliang Li

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；audio-visual(abstract)；分类 cs.CV

Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment

Mustafa Shukor, Guillaume Couairon, Matthieu Cord

专题命中 多模态训练与对齐 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments BMVC 2022

Bootstrapping Multi-view Representations for Fake News Detection

Qichao Ying, Xiaoxiao Hu, Yangming Zhou, Zhenxing Qian, Dan Zeng, Shiming Ge

专题命中 多模态训练与对齐 ：multimodal(abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Authors are from Fudan University, China. Under Review

KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation

Yongfei Liu, Chenfei Wu, Shao-yen Tseng, Vasudev Lal, Xuming He, Nan Duan

专题命中 多模态训练与对齐 ：multi-modal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications

CM-MAE：面向视觉-无线应用的物理引导跨模态自监督学习框架

Yubo Zhang, Yiyao Liu

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV、cs.AI

AI总结 本文提出CM-MAE框架，通过软对比对齐损失等技术实现视觉-无线跨场景表征迁移，在DeepSense 6G数据集上提升了跨场景任务的准确率。