多模态大模型 - arXivDaily 专题

2503.18135 2025-11-11 cs.CV 88%

MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation

Jiaxin Huang, Runnan Chen, Ziwen Li, Zhengqing Gao, Xiao He, Yandong Guo, Mingming Gong, Tongliang Liu

机构 * MBZUAI ； The University of Sydney（悉尼大学）； The University of Melbourne（墨尔本大学）； AI2Robotic

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(title,abstract)；分类 cs.CV

Comments 10 pages, 3 figures

2509.14735 2025-09-19 cs.CL 88%

Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLM

Chenkun Tan, Pengyu Wang, Shaojun Zhou, Botian Jiang, Zhaowei Li, Dong Zhang, Xinghao Wang, Yaqian Zhou, Xipeng Qiu

机构 * Fudan University（复旦大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(title,abstract)；分类 cs.CL

Comments Accepted by Findings of EMNLP2025

2411.02992 2025-09-15 cs.IR cs.CV 88%

Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation

Junchen Fu, Xuri Ge, Xin Xin, Alexandros Karatzoglou, Ioannis Arapakis, Kaiwen Zheng, Yongxin Ni, Joemon M. Jose

机构 * School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）； Amazon（亚马逊）； Telefonica Scientific Research（Telefonica科学研究院）； University of Science and Technology of China（中国科学技术大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；multimodal foundation model(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE)

2508.13843 2025-08-20 cs.IR cs.AI 88%

UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion

Zihan Liang, Yufei Ma, ZhiPeng Qian, Huangyu Dai, Zihan Wang, Ben Chen, Chenyi Lei, Yuqing Ding, Han Li

机构 * Kuaishou Technology（快手科技）

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.AI

Comments Accepted at CIKM2025 as a long paper

2506.01277 2025-06-03 cs.AI 88%

GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models

Qiang Yi, Lianlei Shan

机构 * University of California, Berkeley（加州大学伯克利分校）； Tsinghua University（清华大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；multimodal foundation model(title,abstract)；分类 cs.AI

Comments 29 pages, 14 figures

2505.15233 2025-05-22 cs.CV 88%

CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation

Yuxuan Du, Zhendong Wang, Yuhao Luo, Caiyong Piao, Zhiyuan Yan, Hao Li, Li Yuan

机构 * Peking University（北京大学）； Shenzhen Graduate School（深圳研究生院）； University of Science and Technology of China（中国科学技术大学）； Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.CV

2504.12018 2025-04-17 cs.CV 88%

Instruction-augmented Multimodal Alignment for Image-Text and Element Matching

Xinli Yue, JianHui Sun, Junda Lu, Liangchao Yao, Fan Xia, Tianyi Wang, Fengyun Rao, Jing Lyu, Yuetang Deng

专题命中多模态训练与对齐：multimodal(title,abstract)；image-text(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2025 Workshop

2411.16793 2024-11-27 cs.CV q-bio.GN 88%

ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomics

Yuxiang Lin, Ling Luo, Ying Chen, Xushi Zhang, Zihui Wang, Wenxian Yang, Mengsha Tong, Rongshan Yu

专题命中多模态训练与对齐：multimodal(title,abstract)；multimodal foundation model(title,abstract)；分类 cs.CV

2305.08381 2024-03-01 cs.CV 88%

Parameter-efficient Tuning of Large-scale Multimodal Foundation Model

Haixin Wang, Xinlong Yang, Jianlong Chang, Dian Jin, Jinan Sun, Shikun Zhang, Xiao Luo, Qi Tian

专题命中多模态训练与对齐：multimodal(title,abstract)；multimodal foundation model(title)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS2023

Journal ref Advances in Neural Information Processing Systems 2023

2210.09847 2022-10-19 cs.CV 88%

Multimodal Image Fusion based on Hybrid CNN-Transformer and Non-local Cross-modal Attention

Yu Yuan, Jiaqi Wu, Zhongliang Jing, Henry Leung, Han Pan

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.CV

2207.10851 2022-07-25 cs.CV cs.LG 88%

Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction

Hu Wang, Jianpeng Zhang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, Gustavo Carneiro

专题命中多模态训练与对齐：multi-modal(title,abstract)；cross-modal(title,abstract)；分类 cs.CV

2108.09669 2021-08-24 eess.AS cs.SD 88%

Using Large Pre-Trained Models with Cross-Modal Attention for Multi-Modal Emotion Recognition

Krishna D N

专题命中多模态训练与对齐：cross-modal(title,abstract)；multi-modal(title)；multimodal(abstract)；分类 eess.AS

Comments 5 Pages, REJECTED FROM INTERSPEECH 2021

2511.12449 2026-07-31 cs.CV cs.AI cs.IR cs.LG 版本更新 88%

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

MOON2.0：动态模态平衡多模态表示学习用于电商产品理解

Zhanheng Nie, Chenghan Fu, Daoze Zhang, Junxian Wu, Wanxian Guan, Pengjie Wang, Jian Xu, Bo Zheng

机构 * Alibaba Group（阿里巴巴集团）

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(abstract,abstract_cn)；image-text(abstract)；分类 cs.CV、cs.AI

AI总结 MOON2.0通过动态模态平衡框架解决电商多模态数据中的模态不平衡、信息对齐不足和噪声处理问题，提出MoE、双层对齐和图像-文本共增强策略，提升零样本性能和多模态对齐质量。

Comments Accepted by the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 11 pages, 7 figures

2603.03192 2026-03-31 cs.CV cs.CL cs.LG 88%

MoD-DPO: Towards Mitigating Cross-modal Hallucinations in Omni LLMs using Modality Decoupled Preference Optimization

MoD-DPO：通过模态解耦偏好优化缓解多模态幻觉

Ashutosh Chaubey, Jiacheng Pang, Mohammad Soleymani

机构 * University of Southern California（南加州大学）

专题命中多模态训练与对齐：cross-modal(title,abstract)；multimodal(abstract)；omni-modal(abstract)；multimodal foundation model(abstract)

AI总结本文提出MoD-DPO框架，通过引入模态感知正则化项和语言先验去偏惩罚，提升多模态大模型的模态对齐能力，实验表明其在多模态幻觉基准测试中表现优异。

Comments CVPR 2026. Project Page: https://mod-dpo.github.io/

2412.12718 2025-10-23 cs.CV cs.MM 88%

ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding

Zhenxing Zhang, Yaxiong Wang, Lechao Cheng, Zhun Zhong, Dan Guo, Meng Wang

机构 * School of Computer Science and Information Engineering, Hefei University of Technology, China（计算机科学与信息工程学院，合肥工业大学，中国）； Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China（人工智能研究所，合肥综合性国家科学中心，中国）

专题命中多模态训练与对齐：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；image-text(abstract)

Comments 12 pages, 6 figures

2512.04966 2026-07-24 cs.IT cs.LG eess.SP math.IT 版本更新 88%

Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels

通过跨模态流进行环境感知信道推断：从多模态感知到无线信道

Guangming Liang, Mingjie Yang, Dongzhu Liu, Paul Henderson, Lajos Hanzo

机构 * School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）； School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)

AI总结研究在高多普勒环境下无需导频的信道推断，提出数据驱动框架将传感到信道映射设为跨模态流匹配问题，通过融合多模态特征、学习速度场等实现实时CSI估计，实验表明该方法在信道估计精度和频谱效率上优于基准。

Comments 17 pages, 15 figures, 44 references, to appear in IEEE Transactions on Mobile Computing (TMC)

2602.06409 2026-07-22 cs.CR 版本更新 88%

VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems

VENOMREC: 多模态交互性污染用于多模态大语言模型推荐系统中的定向推广

Guowei Guan, Yurong Hao, Jiaming Zhang, Tiantong Wu, Fuyao Zhang, Tianxiang Chen, Longtao Huang, Cyril Leung, Wei Yang Bryan Lim

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)

AI总结 VENOMREC通过跨模态交互性污染技术，在多模态大语言模型推荐系统中实现定向推广，有效提升推荐性能。

2605.13897 2026-05-15 q-bio.QM cs.LG 88%

Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion

基于注意力的多模态生存预测与跨模态双线性融合

Hassan Keshvarikhojasteh, Josien P. W. Pluim, Mitko Veta

机构 * Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands（埃因霍温理工大学生物医学工程系）

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)

AI总结本文提出一种新型多模态深度学习框架，通过整合全滑片病理特征、RNA测序表达谱和临床变量，利用双线性融合提升参数效率和可解释性，实验表明在CHIMERA数据集上优于拼接基线。

2512.23056 2025-12-30 cs.LG physics.comp-ph 88%

PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations

PI-MFM：基于物理的多模态基础模型用于求解偏微分方程

Min Zhu, Jingmin Sun, Zecheng Zhang, Hayden Schaeffer, Lu Lu

机构 * Department of Statistics and Data Science, Yale University（统计与数据科学系，耶鲁大学）； Department of Applied Mathematics and Statistics, Johns Hopkins University（应用数学与统计学系，约翰霍普金斯大学）； Department of Applied Computational Mathematics and Statistics, University of Notre Dame（应用计算数学与统计学系，圣母大学）； Department of Mathematics, University of California Los Angeles（数学系，加州大学洛杉矶分校）； Department of Chemical and Environmental Engineering, Yale University（化学与环境工程系，耶鲁大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；multimodal foundation model(title,abstract)

AI总结 PI-MFM是一种基于物理的多模态基础模型，通过强制执行偏微分方程在预训练和适应过程中，提高求解PDE的效率和鲁棒性。

2509.09940 2025-09-15 cs.LG 88%

DyKen-Hyena: Dynamic Kernel Generation via Cross-Modal Attention for Multimodal Intent Recognition

Yifei Wang, Wenbin Wang, Yong Luo

机构 * Wuhan University（武汉大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title)；audio-visual(abstract)

Comments 8 pages, 2 figures

2506.10282 2025-06-13 cs.LG 88%

Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning

Jiajin Liu, Dongzhe Fan, Jiacheng Shen, Chuanhao Ji, Daochen Zha, Qiaoyu Tan

机构 * New York University Shanghai（纽约大学上海校区）； University of Chinese Academy of Sciences（中国科学院大学）； Rice University（德克萨斯大学里德学院）

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(title,abstract)

Comments 16 pages, 4 figures

2502.16418 2025-02-25 cs.IT math.IT 88%

M4SC: An MLLM-based Multi-modal, Multi-task and Multi-user Semantic Communication System

Feibo Jiang, Siwei Tu, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan

专题命中多模态训练与对齐：multi-modal(title,abstract)；MLLM(title,abstract)

2407.05420 2024-07-09 cs.IR 88%

Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation

Xinglong Wu, Anfeng Huang, Hongwei Yang, Hui He, Yu Tai, Weizhe Zhang

专题命中多模态训练与对齐：multi-modal(title,abstract)；cross-modal(title,abstract)

2309.01516 2024-02-07 cs.CV cs.AI cs.LG cs.MM 88%

MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrieval

Zijun Long, George Killick, Richard McCreadie, Gerardo Aragon Camarasa

专题命中多模态训练与对齐：multi-modal(title)；image-text(title)；multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

2311.01807 2023-11-06 cs.SI 88%

Cross-modal Consistency Learning with Fine-grained Fusion Network for Multimodal Fake News Detection

Jun Li, Yi Bin, Jie Zou, Jie Zou, Guoqing Wang, Yang Yang

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(title,abstract)

2608.09789 2026-08-11 cs.CV 新提交 87%

ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection

ADOPD：面向多模态大语言模型（MLLM）的工业异常检测的参考特权在线策略蒸馏

Jingtai He, Shiyuan Meng, Wenchao Meng, Qinmin Yang

机构 * Zhejiang University（浙江大学）

专题命中多模态训练与对齐：MLLM(title,title_cn)；multimodal(abstract)；分类 cs.CV

AI总结本研究针对工业异常检测问题，提出ADOPD参考特权在线策略蒸馏框架，通过参考感知教师监督仅查询的学生模型，在MMAD基准零样本推理下获77.31%平均准确率，优于Qwen3-VL-4B主干及其一样本设置。

2604.05695 2026-08-11 cs.CV 版本更新 87%

Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs

让几何引导：在多模态大语言模型中逐层展开几何先验

Chongyu Wang, Ting Huang, Chunyu Sun, Xinyu Ning, Di Wang, Hao Tang

机构 * Xi’an Jiaotong University（西安交通大学）； Peking University（北京大学）； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； InspireOmni

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(abstract,abstract_cn)；cross-modal(abstract)；分类 cs.CV

AI总结本文提出GUIDE框架，通过多级采样和逐步融合几何先验，提升多模态大语言模型在空间推理任务中的性能。

Comments Revised manuscript with updated experiments, figures, and presentation

2510.16598 2026-08-11 cs.CV 版本更新 87%

VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs

VisionSelector：用于高效多模态大语言模型的端到端可学习视觉令牌压缩

Jiaying Zhu, Yurui Zhu, Xin Lu, Wenrui Yan, Dong Li, Kunlin Liu, Xueyang Fu, Zheng-Jun Zha

专题命中多模态训练与对齐：multimodal(title,abstract)；MLLM(abstract,abstract_cn)；分类 cs.CV

AI总结针对多模态大语言模型的视觉令牌压缩瓶颈，提出端到端可学习的VisionSelector框架，通过可微Top-K与课程退火策略实现高效自适应令牌选择，在压缩时保持优异性能并提升预填充速度。

Comments Accepted by ACM Multimedia 2026

2607.27902 2026-08-04 cs.CV 版本更新 87%

One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting

一个补丁就够：面向基于多模态大语言模型（MLLM）的场景文本定位的强化优化视觉标记接地

Rui Tang, Wentao Yang, Peirong Zhang, Yongxin Shi, Shun Zhang, Huiguo He, Lianwen Jin

机构 * South China University of Technology（华南理工大学）； HiThink Research（海思思考研究院）

专题命中多模态训练与对齐：MLLM(title,title_cn)；multimodal(abstract)；分类 cs.CV

AI总结针对现有多补丁场景文本定位范式的冗余噪声与定位歧义问题，提出以视觉为中心的单补丁文本定位框架 SPaTS，通过强化学习优化的单补丁选择等技术实现性能提升，显著优于前沿相关模型。

Comments 15 pages, 11 figures. Accepted to ACM Multimedia 2026

Journal ref Proceedings of the 34th ACM International Conference on Multimedia (MM '26), 2026

2606.24963 2026-06-25 cs.CV cs.LG 新提交 87%

Curvature-Guided Mixing for MLLM Adaptation

曲率引导混合用于MLLM适配

Jinglong Yang, Jiaxuan He, Wenjian Huang, Zhan Zhuang, Jianguo Zhang

机构 * Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学可信自主系统研究院与计算机科学与工程系）； Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）

专题命中多模态训练与对齐：MLLM(title,title_cn)；multimodal(abstract)；分类 cs.CV

AI总结提出曲率引导混合（CGM）框架，通过二阶Hessian近似推导最优软混合比，以及硬混合变体CGM†，在LLaVA-1.5和Qwen2.5VL上改善任务专化与通用知识保持的权衡。

Comments Accepted to ECCV 2026

1. 多模态训练与对齐 6856 篇

MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation

Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLM

Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation

UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion

GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models

CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation

Instruction-augmented Multimodal Alignment for Image-Text and Element Matching

ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomics

Parameter-efficient Tuning of Large-scale Multimodal Foundation Model

Multimodal Image Fusion based on Hybrid CNN-Transformer and Non-local Cross-modal Attention

Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction

Using Large Pre-Trained Models with Cross-Modal Attention for Multi-Modal Emotion Recognition

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

MoD-DPO: Towards Mitigating Cross-modal Hallucinations in Omni LLMs using Modality Decoupled Preference Optimization

ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding

Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels

VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems

Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion

PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations

DyKen-Hyena: Dynamic Kernel Generation via Cross-Modal Attention for Multimodal Intent Recognition

Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning

M4SC: An MLLM-based Multi-modal, Multi-task and Multi-user Semantic Communication System

Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation

MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrieval

Cross-modal Consistency Learning with Fine-grained Fusion Network for Multimodal Fake News Detection

ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection

Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs

VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs

One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting

Curvature-Guided Mixing for MLLM Adaptation