多模态大模型 - arXivDaily 专题

2602.03432 2026-02-04 cs.IR 78%

Failure is Feedback: History-Aware Backtracking for Agentic Traversal in Multimodal Graphs

失败是反馈：面向多模态图中代理遍历的历史感知回溯

Joohyung Yun, Doyup Lee, Wook-Shin Han

专题命中跨模态检索：multimodal(title,abstract)

AI总结 FiF通过历史感知回溯和经济理性的代理工作流程，提升多模态图中检索的适应性和效率。

Comments Project page: https://failureisfeedback.github.io/

2601.18203 2026-01-28 cs.IR 78%

DMAP: Human-Aligned Structural Document Map for Multimodal Document Understanding

DMAP: 人类对齐的多模态文档结构地图用于多模态文档理解

ShunLiang Fu, Yanxin Zhang, Yixin Xiang, Xiaoyu Du, Jinhui Tang

专题命中跨模态检索：multimodal(title,abstract)

AI总结 DMAP通过构建人类对齐的多模态文档结构地图，提升文档理解的结构化表示和多模态推理能力。

Comments WebConf 2026

2601.15434 2026-01-27 cs.CE 78%

ManuRAG: Multi-modal Retrieval Augmented Generation for Manufacturing Question Answering (Early Version)

ManuRAG：面向制造业问答的多模态检索增强生成

Yunqing Li, Zihan Dong, Farhad Ameri, Jianbang Zhang

专题命中跨模态检索：multi-modal(title,abstract)

AI总结 ManuRAG通过多模态检索增强生成技术，提升制造业问答的准确性与可靠性，适用于多种领域。

2601.14714 2026-01-22 cs.IR 78%

Unified Multimodal and Multilingual Retrieval via Multi-Task Learning with NLU Integration

通过多任务学习与NLU集成实现统一的多模态和多语言检索

Xinyuan Zhang, Lina Zhang, Lisung Chen, Guangyao Liu, Shuai Nie, Jiaming Xu, Runyu Shi, Ying Huang, Guoquan Zhang

专题命中跨模态检索：multimodal(title,abstract)

AI总结本文提出通过多任务学习与NLU集成实现统一的多模态和多语言检索，提升跨语言和跨模态的检索效率与准确性。

Comments 4 pages, 2 figures, submitted to IEEE ICASSP 2026

2601.12486 2026-01-21 cs.HC 78%

A Multimodal Assistive System for Product Localization and Retrieval for People who are Blind or have Low Vision

一种多模态辅助系统，用于帮助盲人或低视力者进行产品定位与检索

Ligao Ruan, Giles Hamilton-Fletcher, Mahya Beheshti, Todd E Hudson, Maurizio Porfiri, John-Ross Rizzo

专题命中跨模态检索：multimodal(title,abstract)

AI总结本文提出一种多模态辅助系统，通过目标检测与视觉-语言模型结合，帮助盲人或低视力者自主完成产品定位与检索，提升其自主性和控制感。

2601.11006 2026-01-19 cs.LG 78%

Backdoor Attacks on Multi-modal Contrastive Learning

多模态对比学习中的后门攻击

Simi D Kuniyilh, Rita Machacy

专题命中跨模态检索：multi-modal(title)；multimodal(abstract)

AI总结本文研究了多模态对比学习中的后门攻击问题，分析了攻击方法、威胁模型及防御措施，并探讨了该领域的发展挑战与未来方向。

2508.01346 2026-01-16 cs.CR 78%

MultiCFV: Detecting Control Flow Vulnerabilities in Smart Contracts Leveraging Multimodal Deep Learning

MultiCFV: 利用多模态深度学习检测智能合约中的控制流漏洞

Hongli Peng, Xiaoqi Li, Wenkai Li

专题命中跨模态检索：multimodal(title,abstract)

AI总结 MultiCFV通过融合多模态特征提升智能合约控制流漏洞检测与代码克隆识别的准确性

2508.20209 2025-12-18 physics.med-ph 78%

Low-exposure, high-quality multimodal speckle X-ray imaging via an intrinsic gradient-flow approach

低曝光高质量多模态散射X射线成像：基于内在梯度流方法

Jayvan Liu, Samantha J. Alloo, Max Langer, Konstantin M. Pavlov

专题命中跨模态检索：multimodal(title,abstract)

AI总结本研究提出了一种基于内在梯度流方法的改进算法，用于高效获取高质量的多模态散射X射线成像数据。

Comments 10 pages, 5 figures

2512.14230 2025-12-17 cs.LG stat.ML 78%

Understanding the Gain from Data Filtering in Multimodal Contrastive Learning

理解数据过滤在多模态对比学习中的收益

Divyansh Pareek, Sewoong Oh, Simon S. Du

机构 * Paul G. Allen School of Computer Science and Engineering（保罗·G·艾伦计算机科学与工程学院）

专题命中跨模态检索：multimodal(title,abstract)

AI总结本文研究了数据过滤在多模态对比学习中的收益，通过理论分析证明了过滤能有效降低误差，提升模型性能。

Comments 40 pages, 8 figures, 1 table. This work is accepted to the Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

2512.08995 2025-12-11 cs.HC cs.IR 78%

PoultryTalk: A Multi-modal Retrieval-Augmented Generation (RAG) System for Intelligent Poultry Management and Decision Support

PoultryTalk: 一种用于智能家禽管理与决策支持的多模态检索增强生成（RAG）系统

Kapalik Khanal, Biswash Khatiwada, Stephen Afrifa, Ranjan Sapkota, Sanjay Shah, Frank Bai, Ramesh Bahadur Bist

专题命中跨模态检索：multi-modal(title,abstract)

AI总结 PoultryTalk是一种多模态RAG系统，通过文本和图像交互为家禽管理提供实时专家指导，实现了高准确率和用户满意度。

2512.07216 2025-12-09 cs.IR cs.LG 78%

MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling

MUSE：一种简单而有效的基于多模态搜索的终身用户兴趣建模框架

Bin Wu, Feifan Yang, Zhangming Chan, Yu-Ran Gu, Jiawei Feng, Chao Yi, Xiang-Rong Sheng, Han Zhu, Jian Xu, Mang Ye, Bo Zheng

机构 * Wuhan University（武汉大学）； Alibaba Group（阿里巴巴集团）

专题命中跨模态检索：multimodal(title,abstract)

AI总结 MUSE通过简单有效的多模态搜索框架，实现超长用户行为序列建模，提升推荐系统指标性能。

2512.06449 2025-12-09 cs.IR 78%

Enhancing Medical Cross-Modal Hashing Retrieval using Dropout-Voting Mixture-of-Experts Fusion

通过Dropout投票混合专家融合增强医学跨模态哈希检索

Jaewon Ahn, Woosung Jang, Beakcheol Jang

专题命中跨模态检索：cross-modal(title,abstract)

AI总结本文提出MCMFH模型，通过Dropout投票和混合专家融合提升医疗跨模态哈希检索的准确性和效率。

Comments 5 pages, 1 figure, workshop paper (MMGenSR 2025)

2512.03276 2025-12-04 cs.LG 78%

Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval

太迟回忆：多模态知识检索中双跳问题的解释

Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, Neel Nanda

机构 * University of Oxford（牛津大学）； McGill University（麦吉尔大学）； Meta ； MATS

专题命中跨模态检索：multimodal(title,abstract)

AI总结本文研究了多模态知识检索中双跳问题的影响，发现VLMs在处理视觉输入时过晚解决实体表示导致事实回忆性能下降，并提出通过修补和提示方法恢复性能。

2511.20030 2025-11-26 cs.LG 78%

Cross-Contrastive Clustering for Multimodal Attributed Graphs with Dual Graph Filtering

多模态属性图的跨对比聚类：带有双图过滤的多视图图聚类

Haoran Zheng, Renchi Yang, Hongtao Wang, Jianliang Xu

机构 * Hong Kong Baptist University（香港 Baptist 大学）

专题命中跨模态检索：multimodal(title,abstract)

AI总结本文提出双图过滤方案，通过特征去噪和三交叉对比学习提升多模态属性图的聚类性能。

Comments Accepted by SIGKDD 2026. The code is available at https://github.com/HaoranZ99/DGF

2502.02025 2025-11-26 cs.SE 78%

SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data

SAFE: 借助LLM从多模态碰撞数据中驱动场景的ADS测试

Siwei Luo, Yang Zhang, Yao Deng, Linfeng Liang, Xi Zheng

专题命中跨模态检索：multimodal(title,abstract)

AI总结 SAFE通过多模态提取和LLM技术，提升从碰撞数据中重建真实场景和检测安全违规的能力，实现高准确率和高效生成。

Comments The paper has been accepted for publication in the proceedings of the IEEE/ACM 48th International Conference on Software Engineering to be held 12-18 April 2026 (ICSE2026)

2511.14416 2025-11-19 cs.LG 78%

Toward Robust and Harmonious Adaptation for Cross-modal Retrieval

Haobin Li, Mouxing Yang, Xi Peng

机构 * College of Computer Science, Sichuan University（四川大学计算机学院）； National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University（四川省工程数值模拟基础算法与模型国家重点实验室）

专题命中跨模态检索：cross-modal(title,abstract)

Comments 19 pages, 6 figures

2511.05667 2025-11-11 cs.IR 78%

SARCH: Multimodal Search for Archaeological Archives

Nivedita Sinha, Bharati Khanijo, Sanskar Singh, Priyansh Mahant, Ashutosh Roy, Saubhagya Singh Bhadouria, Arpan Jain, Maya Ramanath

专题命中跨模态检索：multimodal(title)；multi-modal(abstract)

2511.06269 2025-11-11 cs.LG q-bio.QM 78%

LLM$^3$-DTI: A Large Language Model and Multi-modal data co-powered framework for Drug-Target Interaction prediction

Yuhao Zhang, Qinghong Guo, Qixian Chen, Liuwei Zhang, Hongyan Cui, Xiyi Chen

机构 * Polytechnic Institute（多技术学院）； School of Pharmaceutical Sciences（药学学院）； Innovation Center of Yangtze River Delta（长江三角洲创新中心）； Agricultural Genomics Institute at Shenzhen（深圳农业基因组研究院）； School of Public Health（公共卫生学院）

专题命中跨模态检索：multi-modal(title,abstract)

2511.00375 2025-11-04 cs.LG cs.IR 78%

PolyRecommender: A Multimodal Recommendation System for Polymer Discovery

Xin Wang, Yunhao Xiao, Rui Qiao

机构 * Department of Mechanical Engineering（机械工程系）

专题命中跨模态检索：multimodal(title,abstract)

2510.22023 2025-11-04 cs.IR 78%

Multimodal Item Scoring for Natural Language Recommendation via Gaussian Process Regression with LLM Relevance Judgments

Yifan Liu, Qianfeng Wen, Jiazhou Liang, Mark Zhao, Justin Cui, Anton Korikov, Armin Toroghi, Junyoung Kim, Scott Sanner

专题命中跨模态检索：multimodal(title,abstract)

Comments 16 pages,20 figures

2510.27571 2025-11-03 cs.CV cs.AI cs.CL cs.IR cs.LG 78%

Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum

Zhuoning Guo, Mingxin Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Xiaowen Chu

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中跨模态检索：multimodal(title)；分类 cs.CV、cs.CL、cs.AI

2501.15470 2025-11-03 cs.IR cs.MA 78%

CogPlanner: Unveiling the Potential of Agentic Multimodal Retrieval Augmented Generation with Planning

Xiaohan Yu, Zhihan Yang, Chong Chen

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted by SIGIR-AP 2025

2510.22008 2025-10-28 cs.LG q-bio.MN 78%

A Multimodal Human Protein Embeddings Database: DeepDrug Protein Embeddings Bank (DPEB)

Md Saiful Islam Sajol, Magesh Rajasekaran, Hayden Gemeinhardt, Adam Bess, Chris Alvin, Supratik Mukhopadhyay

机构 * Computer Science, Louisiana State University（计算机科学，路易斯安那州立大学）； Computer Science, Furman University（计算机科学，福兰明大学）； Environmental Sciences and Center for Computation and Technology, Louisiana State University（环境科学与计算技术中心，路易斯安那州立大学）

专题命中跨模态检索：multimodal(title,abstract)

2501.04641 2025-10-22 cs.LG math.ST stat.ML stat.TH 78%

A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI

Kazusato Oko, Licong Lin, Yuhang Cai, Song Mei

专题命中跨模态检索：multimodal(title)；multi-modal(abstract)

2509.22369 2025-10-17 cs.LG cs.DC 78%

Role-Aware Multi-modal federated learning system for detecting phishing webpages

Bo Wang, Imran Khan, Martin White, Natalia Beloff

专题命中跨模态检索：multi-modal(title,abstract)

Comments 22 pages, 9 figures

2508.20582 2025-10-13 cs.IR 78%

SUMMA: A Multimodal Large Language Model for Advertisement Summarization

Weitao Jia, Shuo Yin, Zhoufutu Wen, Han Wang, Zehui Dai, Kun Zhang, Zhenyu Li, Tao Zeng, Xiaohui Lv

专题命中跨模态检索：multimodal(title,abstract)

2509.18807 2025-09-24 cs.IR 78%

Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation

Christian Ganhör, Marta Moscati, Anna Hausberger, Shah Nawaz, Markus Schedl

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted by ACM Transactions on Recommender Systems (TORS)

2503.14963 2025-09-23 cs.LG 78%

Continual Multimodal Contrastive Learning

Xiaohao Liu, Xiaobo Xia, See-Kiong Ng, Tat-Seng Chua

机构 * National University of Singapore（新加坡国立大学）

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted by NeurIPS 2025. Codes are available at https://github.com/Xiaohao-Liu/CMCL

2508.01970 2025-08-29 cs.LG 78%

Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modeling

Rituparna Datta, Jiaming Cui, Zihan Guan, Vishal G. Reddy, Joshua C. Eby, Gregory Madden, Rupesh Silwal, Anil Vullikanti

机构 * Department of Computer Science, University of Virginia（大学计算机科学系）； University of Virginia School of Medicine（弗吉尼亚大学医学院）； Virginia Polytechnic Institute and State University（弗吉尼亚理工学院和州立大学）； Biocomplexity Institute and Initiative, University of Virginia（大学生物复杂性研究所）； Division of Infectious Diseases & International Health, University of Virginia School of Medicine（大学感染性疾病与国际卫生分会）

专题命中跨模态检索：multimodal(title,abstract)

2501.15183 2025-08-22 cs.IR 78%

Generating Negative Samples for Multi-Modal Recommendation

Yanbiao Ji, Dan Luo, Chang Liu, Shaokai Wu, Jing Tong, Qicheng He, Deyi Ji, Hongtao Lu, Yue Ding

专题命中跨模态检索：multi-modal(title,abstract)

Comments Accepted by ACM Multimedia

1. 跨模态检索 3454 篇

Failure is Feedback: History-Aware Backtracking for Agentic Traversal in Multimodal Graphs

DMAP: Human-Aligned Structural Document Map for Multimodal Document Understanding

ManuRAG: Multi-modal Retrieval Augmented Generation for Manufacturing Question Answering (Early Version)

Unified Multimodal and Multilingual Retrieval via Multi-Task Learning with NLU Integration

A Multimodal Assistive System for Product Localization and Retrieval for People who are Blind or have Low Vision

Backdoor Attacks on Multi-modal Contrastive Learning

MultiCFV: Detecting Control Flow Vulnerabilities in Smart Contracts Leveraging Multimodal Deep Learning

Low-exposure, high-quality multimodal speckle X-ray imaging via an intrinsic gradient-flow approach

Understanding the Gain from Data Filtering in Multimodal Contrastive Learning

PoultryTalk: A Multi-modal Retrieval-Augmented Generation (RAG) System for Intelligent Poultry Management and Decision Support

MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling

Enhancing Medical Cross-Modal Hashing Retrieval using Dropout-Voting Mixture-of-Experts Fusion

Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval

Cross-Contrastive Clustering for Multimodal Attributed Graphs with Dual Graph Filtering

SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data

Toward Robust and Harmonious Adaptation for Cross-modal Retrieval

SARCH: Multimodal Search for Archaeological Archives

LLM$^3$-DTI: A Large Language Model and Multi-modal data co-powered framework for Drug-Target Interaction prediction

PolyRecommender: A Multimodal Recommendation System for Polymer Discovery

Multimodal Item Scoring for Natural Language Recommendation via Gaussian Process Regression with LLM Relevance Judgments

Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum

CogPlanner: Unveiling the Potential of Agentic Multimodal Retrieval Augmented Generation with Planning

A Multimodal Human Protein Embeddings Database: DeepDrug Protein Embeddings Bank (DPEB)

A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI

Role-Aware Multi-modal federated learning system for detecting phishing webpages

SUMMA: A Multimodal Large Language Model for Advertisement Summarization

Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation

Continual Multimodal Contrastive Learning

Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modeling

Generating Negative Samples for Multi-Modal Recommendation