多模态大模型

Token Activation Map to Visually Explain Multimodal LLMs

Yi Li, Hualiang Wang, Xinpeng Ding, Haonan Wang, Xiaomeng Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV2025 Accepted

Multimodal Prompt Alignment for Facial Expression Recognition

Fuyan Ma, Yiran He, Bin Sun, Shutao Li

机构 * Chinese Academy of Military Science（中国军事科学院） ； Changchun University of Science and Technology（长春理工大学） ； Hunan University（湖南大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments To appear in ICCV2025

SAE-V: Interpreting Multimodal Models for Enhanced Alignment

Hantao Lou, Changye Li, Jiaming Ji, Yaodong Yang

机构 * Institute for AI, Peking University, Beijing, China（人工智能研究院，北京大学，北京，中国） ； State Key Laboratory of General Artificial Intelligence, Institute for AI, Peking University, Beijing, China（通用人工智能国家重点实验室，人工智能研究院，北京大学，北京，中国） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 13 figures

Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

Yi Peng, Peiyu Wang, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu, Yahui Zhou

机构 * Skywork AI（Skywork人工智能） ； Kunlun Inc.（昆仑公司） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Towards Achieving Perfect Multimodal Alignment

Abhi Kamboj, Minh N. Do

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learning

Xudong Yan, Songhe Feng, Yang Zhang, Jian Yang, Yueguan Lin, Haojun Fei

机构 * School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院，北京交通大学） ； Qifu Technology（启福科技） 纠错

专题命中 多模态训练与对齐 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IJCAI 2025

Enhancing Multimodal Continual Instruction Tuning with BranchLoRA

Duzhen Zhang, Yong Ren, Zhong-Zhi Li, Yahan Yu, Jiahua Dong, Chenxing Li, Zhilong Ji, Jinfeng Bai

机构 * Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed大学人工智能学院） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； Kyoto University（京都大学） ； Tencent AI Lab（腾讯AI实验室） ； Tomorrow Advancing Life（明天生命科技） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL2025 Main Conference

SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings

Weikai Lu, Hao Peng, Huiping Zhuang, Cen Chen, Ziqian Zeng

机构 * South China University of Technology, China（华南理工大学） ； Beihang University, China（北航） ； Pazhou Laboratory, China（琶洲实验室） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.MM

Comments Accepted in ACL 2025 Main Track

Enhancing Biomedical Multi-modal Representation Learning with Multi-scale Pre-training and Perturbed Report Discrimination

Xinliu Zhong, Kayhan Batmanghelich, Li Sun

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments 6 pages, 1 figure, accepted by 2024 IEEE Conference on Artificial Intelligence (CAI)

Journal ref 2024 IEEE Conference on Artificial Intelligence (CAI), 2024, 480-485

Enhancing Multimodal Unified Representations for Cross Modal Generalization

Hai Huang, Yan Xia, Shengpeng Ji, Shulei Wang, Hanting Wang, Minghui Fang, Jieming Zhu, Zhenhua Dong, Sashuai Zhou, Zhou Zhao

机构 * Zhejiang University（浙江大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts

Xin He, Xumeng Han, Longhui Wei, Lingxi Xie, Qi Tian

机构 * Huawei Inc.（华为公司） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?

Zichen Wen, Yifeng Gao, Weijia Li, Conghui He, Linfeng Zhang

机构 * Shanghai Jiao Tong University（上海交通大学） ； EPIC Lab, SJTU（EPIC实验室，SJTu） ； Shanghai AI Laboratory（上海人工智能实验室） ； Sun Yat-sen University（中山大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2025 Findings

Multi-MLLM Knowledge Distillation for Out-of-Context News Detection

Yimeng Gu, Zhao Tong, Ignacio Castro, Shu Wu, Gareth Tyson

机构 * Queen Mary University of London（伦敦玛丽女王大学） ； Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； The Hong Kong University of Science and Technology (GZ)（香港科学与技术大学） 纠错

专题命中 多模态训练与对齐 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.MM

Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval

Hailong Ning, Siying Wang, Tao Lei, Xiaopeng Cao, Huanmin Dou, Bin Zhao, Asoke K. Nandi, Petia Radeva

专题命中 多模态训练与对齐 ：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture

Nischal Mandal, Yang Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Adaptive Token Boundaries: Integrating Human Chunking Mechanisms into Multimodal LLMs

Dongxing Yu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

SSLFusion: Scale & Space Aligned Latent Fusion Model for Multimodal 3D Object Detection

Bonan Ding, Jin Xie, Jing Nie, Jiale Cao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI 2025

Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning

Ji Hyeok Jung, Eun Tae Kim, Seoyeon Kim, Joo Ho Lee, Bumsoo Kim, Buru Chang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments CVPR2025 Camera-ready

Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training

Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jiawen Liu, Jifeng Dai, Yu Qiao, Xizhou Zhu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Accepted by CVPR 2025

Lightweight Multimodal Artificial Intelligence Framework for Maritime Multi-Scene Recognition

Xinyu Xi, Hua Yang, Shentai Zhang, Yijie Liu, Sijin Sun, Xiuju Fu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 19 pages, 4 figures, submitted to Engineering Applications of Artificial Intelligence

Large Language Model Guided Progressive Feature Alignment for Multimodal UAV Object Detection

Wentao Wu, Chenglong Li, Xiao Wang, Bin Luo, Qi Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices

Xudong Lu, Yinghao Chen, Renshou Wu, Haohao Gao, Xi Chen, Xue Yang, Xiangyu Zhao, Aojun Zhou, Fangyuan Li, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 14 pages

Developing a PET/CT Foundation Model for Cross-Modal Anatomical and Functional Imaging

Yujin Oh, Robert Seifert, Yihan Cao, Christoph Clement, Justin Ferdinandus, Constantin Lapa, Alessandro Liebich, Michelle Amon, Johanna Enke, Sifan Song, Runqi Meng, Fang Zeng, Ning Guo, Xiang Li, Pedram Heidari, Axel Rominger, Kuangyu Shi, Quanzheng Li

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 2 figures, 3 tables

Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

Min Shi, Fuxiao Liu, Shihao Wang, Shijia Liao, Subhashree Radhakrishnan, Yilin Zhao, De-An Huang, Hongxu Yin, Karan Sapra, Yaser Yacoob, Humphrey Shi, Bryan Catanzaro, Andrew Tao, Jan Kautz, Zhiding Yu, Guilin Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Github: https://github.com/NVlabs/Eagle, HuggingFace: https://huggingface.co/NVEagle

All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment

Chunhui Zhang, Xin Sun, Yiqian Yang, Li Liu, Qiong Liu, Xi Zhou, Yanfeng Wang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments In this version, we corrected some typos

MM-RLHF: The Next Step Forward in Multimodal LLM Alignment

Yi-Fan Zhang, Tao Yu, Haochen Tian, Chaoyou Fu, Peiyan Li, Jianshu Zeng, Wulin Xie, Yang Shi, Huanyu Zhang, Junkang Wu, Xue Wang, Yibo Hu, Bin Wen, Fan Yang, Zhang Zhang, Tingting Gao, Di Zhang, Liang Wang, Rong Jin, Tieniu Tan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://mm-rlhf.github.io/

LLMI3D: MLLM-based 3D Perception from a Single 2D Image

Fan Yang, Sicheng Zhao, Yanhao Zhang, Hui Chen, Haonan Lu, Jungong Han, Guiguang Ding

专题命中 多模态训练与对齐 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

A Novel Approach to for Multimodal Emotion Recognition : Multimodal semantic information fusion

Wei Dai, Dequan Zheng, Feng Yu, Yanrong Zhang, Yaohui Hou

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

MapFusion: A Novel BEV Feature Fusion Network for Multi-modal Map Construction

Xiaoshuai Hao, Yunfeng Diao, Mengchuan Wei, Yifan Yang, Peng Hao, Rong Yin, Hui Zhang, Weiming Li, Shu Zhao, Yu Liu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Virgo: A Preliminary Exploration on Reproducing o1-like MLLM

Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, Ji-Rong Wen

专题命中 多模态训练与对齐 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Technical Report on Slow Thinking with LLMs: Visual Reasoning