arXivDaily arXiv每日学术速递 周一至周五更新

AI 大模型

多模态大模型

跨文本、图像、视频、音频等模态的大模型与学习方法。

共收录 6856 信号源:cs.CV, cs.CL, cs.AI, cs.MM, eess.AS

1. 多模态训练与对齐 6856 篇

2403.15356 2025-10-17 cs.CV 86%

Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation

Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J. Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, Xiao Xiang Zhu

机构 * Chair of Data Science in Earth Observation, Technical University of Munich (TUM)(地球观测数据科学教授职位,慕尼黑技术大学) Munich Center for Machine Learning(慕尼黑机器学习中心) AIML Lab, School of Computer Science, University of St. Gallen(人工智能实验室,圣加尔登大学计算机科学学院) School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens(农村、测绘与地理信息工程学院,国家技术大学雅典) Image Processing Laboratory (IPL), Universitat de València(图像处理实验室(IPL),瓦伦西亚大学)

专题命中 多模态训练与对齐 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.CV

Comments 18 pages, 8 figures

详情

展开后加载摘要…

URL PDF HTML 收藏
2509.24896 2025-09-30 cs.CV 86%

DAM: Dual Active Learning with Multimodal Foundation Model for Source-Free Domain Adaptation

Xi Chen, Hongxun Yao, Zhaopan Xu, Kui Jiang

机构 * Harbin Institute of Technology(哈尔滨工业大学)

专题命中 多模态训练与对齐 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.CV

Comments 5 pages

详情

展开后加载摘要…

URL PDF HTML 收藏
2412.19237 2025-04-22 cs.CV cs.LG 86%

SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing

Xuyang Li, Chenyu Li, Gemine Vivone, Danfeng Hong

机构 * Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院 aerospace information research institute) School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院) School of Mathematics and Statistics, Southeast University(东南大学数学与统计学院) Institute of Methodologies for Environmental Analysis, National Research Council, CNR-IMAA(国家研究理事会环境分析方法研究所) Graduate School of Frontier Sciences, the University of Tokyo(东京大学前沿科学研究生院)

专题命中 多模态训练与对齐 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2502.14044 2025-02-26 cs.CV cs.LG 86%

Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data

Yucheng Shi, Quanzheng Li, Jin Sun, Xiang Li, Ninghao Liu

专题命中 多模态训练与对齐 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.CV

Comments Accepted by ICLR 2025. Code: https://github.com/sycny/SelfSynthX

详情

展开后加载摘要…

URL PDF HTML 收藏
2501.15063 2025-01-28 cs.CL 86%

Cross-modal Context Fusion and Adaptive Graph Convolutional Network for Multimodal Conversational Emotion Recognition

Junwei Feng, Xueyan Fan

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(title);分类 cs.CL

详情

展开后加载摘要…

URL PDF HTML 收藏
2410.01408 2024-10-03 cs.CV 86%

SHAP-CAT: A interpretable multi-modal framework enhancing WSI classification via virtual staining and shapley-value-based multimodal fusion

Jun Wang, Yu Mao, Nan Guan, Chun Jason Xue

专题命中 多模态训练与对齐 :multimodal(title,abstract);multi-modal(title);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2401.16936 2024-01-31 cs.LG cs.CV 86%

Multi-modal Representation Learning for Cross-modal Prediction of Continuous Weather Patterns from Discrete Low-Dimensional Data

Alif Bin Abdul Qayyum, Xihaier Luo, Nathan M. Urban, Xiaoning Qian, Byung-Jun Yoon

专题命中 多模态训练与对齐 :multi-modal(title,abstract);cross-modal(title);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2304.14381 2023-05-18 cs.CV 86%

$π$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation

Chengyue Wu, Teng Wang, Yixiao Ge, Zeyu Lu, Ruisong Zhou, Ying Shan, Ping Luo

专题命中 多模态训练与对齐 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.CV

Comments To appear in ICML 2023

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.11167 2026-08-12 cs.CV cs.CL cs.LG 新提交 86%

MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment

多模态代码切换:将视觉对象交织入语言以实现显式对象级对齐

Changhao Xiang, Shangyu Xing, Zhen Wu, Jianbing Zhang, Xinyu Dai

机构 * National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

专题命中 多模态训练与对齐 :multimodal(title,abstract);image-text(abstract);分类 cs.CV、cs.CL

AI总结 针对现有多模态大语言模型的图像级对齐存在指称歧义的问题,提出多模态代码切换(MMCS)范式,构建含77.3万样本的数据集,仅用5万样本即可匹配或超越60万图像-文本对训练的模型,提升了视觉基础与感知能力。

详情

展开后加载摘要…

URL PDF HTML 收藏
2606.03793 2026-08-12 cs.CL cs.CV 版本更新 86%

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

探索多语言多模态大语言模型的对抗鲁棒性与安全对齐

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

机构 * Mohamed Bin Zayed University of AI, UAE(穆罕默德·本·扎耶德人工智能大学,阿联酋) Khalifa University, UAE(卡比拉大学,阿联酋) Australian National University, Australia(澳大利亚国立大学,澳大利亚)

专题命中 多模态训练与对齐 :multi-modal(title);MLLM(abstract,abstract_cn);multimodal(abstract);分类 cs.CV、cs.CL

AI总结 本研究通过梯度攻击和跨语言评估,发现多语言多模态大语言模型存在可迁移的对抗脆弱性,并揭示低资源语言因理解失败而呈现的虚假安全现象,提出深层训练整合才能实现真正的多语言安全对齐。

Journal ref The 37th British Machine Vision Conference (BMVC) 2026

详情

展开后加载摘要…

URL PDF HTML 收藏
2603.05997 2026-08-07 cs.CV cs.AI 版本更新 86%

MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMs

MM-ISTS: 基于多模态视觉-文本大语言模型的不规则采样时间序列预测

Zhi Lei, Chenxi Liu, Hao Miao, Wanghui Qiu, Bin Yang, Chenjuan Guo

机构 * East China Normal University Shanghai China Centre for Artificial Intelligence Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences Hong Kong China Department of Computing, The Hong Kong Polytechnic University Hong Kong China East China Normal University Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences Department of Computing, The Hong Kong Polytechnic University

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract);cross-modal(abstract);分类 cs.CV、cs.AI

AI总结 MM-ISTS利用多模态视觉-文本大语言模型,通过双阶段编码机制提升不规则采样时间序列预测的性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2604.00513 2026-08-06 cs.LG cs.AI cs.CV cs.IR 版本更新 86%

MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding

MOON3.0: 基于推理的多模态表示学习用于电商产品理解

Junxian Wu, Chenghan Fu, Zhanheng Nie, Daoze Zhang, Bowen Wan, Wanxian Guan, Chuan Yu, Jian Xu, Bo Zheng

机构 * Alibaba Group(阿里巴巴集团)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.AI

AI总结 MOON3.0通过多头模态融合、联合对比与强化学习框架及细粒度残差增强模块,提升电商产品细粒度属性建模能力,并在大规模多模态电商基准MBe3.0上实现零样本最优性能。

Comments Accepted by the 34th ACM International Conference on Multimedia (ACM MM), 2026. 10 pages, 6 figures

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.03611 2026-08-05 cs.AI cs.MM 新提交 86%

Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observations

面向含不完整观测的多模态情感分析,重新思考模态可靠性

Chunlei Meng, Jacqueline J. Pang, Pengbin Feng, Zhenyu Yu, Chun Ouyang, Zhongxue Gan

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(abstract);分类 cs.AI、cs.MM

AI总结 针对含不完整观测的多模态情感分析,本文提出显式建模模态可靠性的MRCF框架,缓解可靠性不匹配与传播偏差,在多个公开情感数据集上取得优异性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.22043 2026-07-27 cs.CL cs.CV 新提交 86%

Scaling Native Multimodal Pre-Training From Scratch

从零开始扩展原生多模态预训练

Haoyuan Wu, Aoqi Wu, Hai Wang, Jiajia Wu, Jinxiang Ou, Bei Yu

机构 * The Chinese University of Hong Kong(香港中文大学) LLM Department, Tencent(腾讯大语言模型部)

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(abstract);multimodal foundation model(abstract);分类 cs.CV、cs.CL

AI总结 研究在固定计算预算下训练基于Transformer的视觉语言模型的最优模型大小和token数量,发现语言和多模态目标扩展行为不同,推导效率前沿,还表明原生多模态预训练能促进跨模态转移,为扩展多模态基础模型奠定基础。

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.12112 2026-07-15 cs.LG cs.AI cs.CV cs.DC 新提交 86%

Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning

用于联邦多模态大语言模型微调的弹性正则化和合成重放持续学习

Jing Liu, Chenxuanyin Zou, Jiayang Ren, Gaoyun Fang, Chengfang Li, Yan Wang, Zhenchao Ma, Bo Hu

机构 * The University of British Columbia(英属哥伦比亚大学) Fudan University(复旦大学) Royal College of Science, Imperial College London(伦敦帝国理工学院皇家科学学院) Dyson School of Design Engineering(戴森设计工程学院) Suzhou Institute of Biomedical Engineering and Technology (SIBET), Chinese Academy of Sciences(中国科学院苏州生物医学工程技术研究所) East China Normal University(华东师范大学)

专题命中 多模态训练与对齐 :MLLM(title,abstract_cn);multimodal(abstract);cross-modal(abstract);分类 cs.CV、cs.AI

AI总结 研究针对联邦多模态大语言模型微调中灾难性遗忘问题,提出FedCMM框架,在参数、数据、聚合三个层面嵌入持续学习保障,经实验验证该框架在准确性和反向迁移上优于基线,能实现跨异构网络AI部署的稳健进化适应。

Comments submitted to IEEE JSTSP

详情

展开后加载摘要…

URL PDF HTML 收藏
2604.22823 2026-07-14 cs.CV cs.AI 版本更新 86%

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

PivotMerge: 通过后对齐模型融合弥合异构多模态预训练

Zibo Shao, Baochen Xiong, Xiaoshan Yang, Yaguang Song, Qimeng Zhang, Haifeng Chen, Changsheng Xu

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所) Pengcheng Laboratory(鹏城实验室) School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院) Data Science and Artificial Intelligence Research Institute, China United Network Communications Group Co., Ltd.(中国联合网络通信集团有限公司数据科学与人工智能研究院)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract);cross-modal(abstract);分类 cs.CV、cs.AI

AI总结 本文提出PivotMerge框架,通过后对齐模型融合技术,解决多模态预训练中跨模态对齐能力整合问题,提升模型统一语义空间的构建效果。

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.00746 2026-07-02 cs.CV cs.AI 新提交 86%

GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perception

GaussianFusion:用于多模态融合感知的统一3D高斯表示

Xiao Zhao, Chang Liu, Mingxu Zhu, Zheyuan Zhang, Linna Song, Qingliang Luo, Chufan Guo, Kuifeng Su

机构 * Tencent, Autonomous Driving Lab(腾讯自动驾驶实验室)

专题命中 多模态训练与对齐 :multi-modal(title,abstract);multimodal(abstract);cross-modal(abstract);分类 cs.CV、cs.AI

AI总结 提出基于3D高斯表示的多模态融合框架GaussianFusion,通过连续3D高斯空间统一多模态特征,保留细节,支持多种3D感知任务,在nuScenes上检测NDS提升2.6,占用预测mIoU提升1.55且速度提升450%。

Comments ICLR 2026

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.00293 2026-07-02 cs.CV cs.CL cs.LG 新提交 86%

Rosetta: Composable Native Multimodal Pretraining

Rosetta: 可组合的原生多模态预训练

Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Ping Tan

机构 * HKUST(香港科技大学) Tencent Hunyuan(腾讯混元)

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(abstract);multimodal foundation model(abstract);分类 cs.CV、cs.CL

AI总结 提出Rosetta框架,通过动量锚定正交投影(MAOP)实现无损模态扩展,解决多模态预训练中的灾难性遗忘和梯度冲突问题。

详情

展开后加载摘要…

URL PDF HTML 收藏
2606.31903 2026-07-01 cs.CV cs.AI 新提交 86%

Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference

关注、变换或静默:面向高效多模态大语言模型推理的算子级视觉跳跃

Zhaoyang Luo, Runmin Dong, Miao Yang, Fan Wei, Yushan Lai, Bin Luo, Haohuan Fu

机构 * Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院) Sun Yat-sen University(中山大学) National Supercomputing Center in Shenzhen(国家超级计算深圳中心) Tsinghua University(清华大学)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.AI

AI总结 针对多模态大语言模型视觉令牌计算冗余问题,提出算子级视觉跳跃框架,选择性跳过冗余注意力或FFN算子,在Qwen3-VL上减少33.7%计算量并保持99.5%性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2606.17791 2026-06-17 cs.CL cs.CV 新提交 86%

The Slop Paradox: How Synthetic Standardization Erodes Clinical Uncertainty and Cross-Modal Alignment in AI-Rewritten Radiology Reports

Slop悖论:合成标准化如何侵蚀AI重写放射学报告中的临床不确定性和跨模态对齐

Samar Ansari

机构 * School of Computing and Engineering Sciences, University of Chester(切斯特大学计算与工程科学学院)

专题命中 多模态训练与对齐 :cross-modal(title,abstract);multimodal(abstract);image-text(abstract);分类 cs.CV、cs.CL

AI总结 本研究通过控制实验测量AI重写放射学报告导致的信息退化,发现电子健康记录摘要虽破坏内容但保留图像-文本对齐,而标准化重写和教学病例准备则相反,造成更大对齐损失,称为slop悖论。

详情

展开后加载摘要…

URL PDF HTML 收藏
2606.15880 2026-06-16 cs.CV cs.AI 新提交 86%

Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

深度残差注入:多模态大语言模型的全频谱取证信号感知

Kaiqing Lin, Zhiyuan Yan, Ruoxin Chen, Ke-Yue Zhang, Yue Zhou, Caiyong Piao, Bin Li, Taiping Yao, Bo Wang, Youchang Xiao, Shouhong Ding

机构 * National University of Singapore(新加坡国立大学) Tsinghua University(清华大学) University of Science and Technology of China(中国科学技术大学) University of Electronic Science and Technology of China(电子科技大学) University of California, Berkeley(加州大学伯克利分校)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.AI

AI总结 针对多模态大语言模型在取证中难以同时保留语义知识和捕获低级生成器伪影的问题,提出Deep-VRM方法,通过将伪影特定视觉信号作为残差路径注入中间层,实现全频谱信号感知,达到鲁棒检测性能。

Comments Accepted at ICML 2026

详情

展开后加载摘要…

URL PDF HTML 收藏
2603.20020 2026-05-27 cs.CV cs.AI 86%

Detached Skip-Links and $R$-Probe: Decoupling Feature Aggregation from Gradient Propagation for MLLM OCR

分离跳跃链接与$R$-探针:解耦特征聚合与梯度传播用于MLLM OCR

Ziye Yuan, Ruchang Yao, Chengxin Zheng, Yusheng Zhao, Daxiang Dong, Ming Zhang

机构 * State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems, Peking University, Beijing, China(多媒体信息处理国家重点实验室,计算机科学学院,PKU-Anker LLM实验室,软件与硬件协同人工智能系统北京重点实验室,北京大学,北京,中国) Tsinghua University, Beijing, China(清华大学,北京,中国) Baidu Inc, Beijing, China(百度公司,北京,中国)

专题命中 多模态训练与对齐 :MLLM(title,title_cn);multimodal(abstract);分类 cs.CV、cs.AI

AI总结 针对多模态大语言模型在OCR任务中因梯度干扰导致细粒度视觉信息丢失的问题,提出分离跳跃链接(Detached Skip-Links)以解耦前向特征聚合与反向梯度传播,并引入$R$-探针($R$-Probe)诊断视觉令牌的可重构性,从而提升OCR及通用多模态任务性能。

Comments Accepted by ICML 2026. Ziye Yuan and Ruchang Yao contributed equally to this work (co-first authors, listed in random order)

详情

展开后加载摘要…

URL PDF HTML 收藏
2605.26110 2026-05-26 cs.LG cs.CL cs.CV 86%

Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning

Prism:面向可扩展多模态持续指令微调的插件式可复现基础设施

Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie, Da-Wei Zhou

机构 * School of Artificial Intelligence, Nanjing University, China(南京大学人工智能学院) National Key Laboratory for Novel Software Technology, Nanjing University, China(南京大学新型软件技术国家重点实验室)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.CL

AI总结 针对多模态持续指令微调中工程瓶颈问题,提出Prism插件式代码库,通过轻量级插件注册机制分离算法开发与骨干实现,支持大规模训练流水线,实现可复现、可扩展的实验。

Comments Code is available at https://github.com/LAMDA-CL/Prism

详情

展开后加载摘要…

URL PDF HTML 收藏
2605.12374 2026-05-26 cs.CV cs.AI cs.LG 86%

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

填补GAP:多模态大语言模型中视觉推理的粒度对齐范式

Yanting Miao, Yutao Sun, Dexin Wang, Mengyu Zhou, Pascal Poupart, Lei Lv, Qi Zhao, Li Wang, Hao Li, Xiaoxi Jiang, Guanjun Jiang

机构 * Qwen Large Model Application Team, Alibaba(阿里云大模型应用团队) Alibaba University of Waterloo(阿里大学水力学院) Vector Institute(向量研究所) Zhejiang University(浙江大学)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.AI

AI总结 提出GAP(粒度对齐范式),通过特征级、上下文级和能力引导级对齐,解决多模态大语言模型中视觉潜在推理的特征空间不匹配问题,提升感知与推理性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2603.23404 2026-04-21 cs.CV cs.CL 86%

Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning

通过文本表示引导推理来解锁多模态大语言模型中的空间推理

Jiacheng Hua, Yishu Yin, Yuhang Wu, Tai Wang, Yifei Huang, Miao Liu

机构 * College of AI, Tsinghua University, Beijing, China(清华大学人工智能学院,北京,中国) Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室,上海,中国) The University of Tokyo, Tokyo, Japan(东京大学,东京,日本)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.CL

AI总结 本文提出TRACE方法,通过生成文本表示来提升多模态大语言模型的空间推理能力,实验表明其在多个基准测试中表现优异。

Comments Accepted to ACL 2026. 22 pages, 6 figures, 10 tables. Project page: https://trace-reasoning.github.io

详情

展开后加载摘要…

URL PDF HTML 收藏
2604.16462 2026-04-21 cs.CV cs.AI 86%

From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration

从继承到饱和:解构视觉冗余的演化以实现架构感知的MLLM推理加速

Jiaqi Shi, Yuechan Li, Xulong Zhang, Xiaoyang Qu, Jianzong Wang

机构 * University of Science and Technology of China(中国科学技术大学) Wuhan University(武汉大学) Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)

专题命中 多模态训练与对齐 :MLLM(title,title_cn);multimodal(abstract);分类 cs.CV、cs.AI

AI总结 本文提出HalfV框架,通过统一剪枝策略缓解内在视觉冗余,并根据具体表现适应性处理二次饱和冗余,实现跨架构的高效推理。

Comments 16 pages, 14 figures, plus appendix, accepted at ACL 2026

详情

展开后加载摘要…

URL PDF HTML 收藏
2604.12767 2026-04-15 cs.CV cs.AI 86%

CLASP: Class-Adaptive Layer Fusion and Dual-Stage Pruning for Multimodal Large Language Models

CLASP:面向多模态大语言模型的类适应层融合与双阶段剪枝

Yunkai Dang, Yizhu Jiang, Yifan Jiang, Qi Fan, Yinghuan Shi, Wenbin Li, Yang Gao

机构 * School of Artificial Intelligence Science and Technology(人工智能科学与技术学院)

专题命中 多模态训练与对齐 :multimodal(title,abstract);MLLM(abstract,abstract_cn);分类 cs.CV、cs.AI

AI总结 CLASP通过类适应层融合和双阶段剪枝,实现多模态大语言模型中视觉token的高效减少,提升模型鲁棒性和性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2601.07516 2026-04-14 cs.CL cs.AI cs.LG 86%

Controlling Multimodal Conversational Agents with Coverage-Enhanced Latent Actions

通过增强覆盖的潜在动作控制多模态对话代理

Yongqi Li, Hao Lang, Tieyun Qian, Yongbin Li

机构 * School of Computer Science, Wuhan University(武汉大学计算机学院) Tongyi Lab(通义实验室) Zhongguancun Academy(中关村学院)

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(abstract);image-text(abstract);分类 cs.CL、cs.AI

AI总结 本文提出通过增强覆盖的潜在动作空间改进多模态对话代理的强化学习微调,利用跨模态投影器提升鲁棒性,在两个对话任务中优于基线方法。

Comments Accepted to ACL 2026 (Main), camera-ready version

详情

展开后加载摘要…

URL PDF HTML 收藏
2505.11404 2026-03-24 cs.CV cs.AI 86%

Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner

Patho-R1: 基于多模态强化学习的病理专家推理器

Wenchuan Zhang, Penghao Zhang, Jingru Guo, Tao Cheng, Jie Chen, Shuwan Zhang, Zhang Zhang, Yuhao Yi, Hong Bu

机构 * Department of Pathology, West China Hospital, Sichuan University(四川大学华西医院病理科部门) Institute of Clinical Pathology, West China Hospital, Sichuan University(四川大学华西医院临床病理科研究所) University of Toronto(多伦多大学) Business School, Sichuan University(四川大学商学院) Department of Pathology, Shengjing Hospital of China Medical University(中国医科大学盛京医院病理科部门)

专题命中 多模态训练与对齐 :multimodal(title,abstract);cross-modal(abstract);image-text(abstract);分类 cs.CV、cs.AI

AI总结 本文提出Patho-R1,通过构建高质量推理导向数据集,结合三阶段训练流程提升病理推理能力,实现跨模态任务的鲁棒性能。

Journal ref Proceedings of the AAAI Conference on Artificial Intelligence, 40(33): 28418-28426, 2026

详情

展开后加载摘要…

URL PDF HTML 收藏
2509.14671 2026-02-12 cs.CL cs.AI cs.LG 86%

TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding

TableDART: 表格理解的动态自适应多模态路由

Xiaobo Xing, Wei Yuan, Tong Chen, Quoc Viet Hung Nguyen, Xiangliang Zhang, Hongzhi Yin

机构 * The University of Queensland, Australia(昆士兰大学) Griffith University, Australia(格里菲斯大学) University of Notre Dame, USA(诺丁汉大学)

专题命中 多模态训练与对齐 :multi-modal(title);multimodal(abstract);MLLM(abstract);cross-modal(abstract)

AI总结 TableDART通过动态选择文本、图像或融合视角,提升表格理解的准确性和效率,避免昂贵的多模态模型微调。

Comments Accepted to ICLR 2026. 26 pages, 11 figures

详情

展开后加载摘要…

URL PDF HTML 收藏