多模态大模型

D$^2$TV: Dual Knowledge Distillation and Target-oriented Vision Modeling for Many-to-Many Multimodal Summarization

Yunlong Liang, Fandong Meng, Jiaan Wang, Jinan Xu, Yufeng Chen, Jie Zhou

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2023 Findings

Emotion recognition based on multi-modal electrophysiology multi-head attention Contrastive Learning

Yunfei Guo, Tao Zhang, Wu Huang

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments arXiv admin note: text overlap with arXiv:2208.00877 by other authors

MolFM: A Multimodal Molecular Foundation Model

Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, Zaiqing Nie

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)

Comments 31 pages, 15 figures, and 15 tables

Multi-Modal Self-Supervised Learning for Recommendation

Wei Wei, Chao Huang, Lianghao Xia, Chuxu Zhang

专题命中 跨模态检索 ：multi-modal(title,abstract)；cross-modal(abstract)

Comments This paper has been published as a full paper at WWW 2023

Grounding Language Models to Images for Multimodal Inputs and Outputs

Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Published in ICML 2023. Project page: https://jykoh.com/fromage

SemiMemes: A Semi-supervised Learning Approach for Multimodal Memes Analysis

Pham Thai Hoang Tung, Nguyen Tan Viet, Ngo Tien Anh, Phan Duy Hung

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ICCCI 2023

Is Cross-modal Information Retrieval Possible without Training?

Hyunjin Choi, Hyunjae Lee, Seongho Joe, Youngjune L. Gwon

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, Proceedings, Part II

VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval

Siteng Huang, Biao Gong, Yulin Pan, Jianwen Jiang, Yiliang Lv, Yuyuan Li, Donglin Wang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by CVPR 2023

Mr. Right: Multimodal Retrieval on Representation of ImaGe witH Text

Cheng-An Hsieh, Cheng-Ping Hsieh, Pu-Jen Cheng

专题命中 跨模态检索 ：multimodal(title,abstract)；image-text(abstract)

Comments Dataset available at https://github.com/hsiehjackson/Mr.Right

Multimodal Entity Tagging with Multimodal Knowledge Base

Hao Peng, Hang Li, Lei Hou, Juanzi Li, Chao Qiao

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 11 pages, 4 figures

Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss

Riccardo Franceschini, Enrico Fini, Cigdem Beyan, Alessandro Conti, Federica Arrigoni, Elisa Ricci

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to 26th International Conference on Pattern Recognition (ICPR) 2022

Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue

Jiudong Yang, Peiying Wang, Yi Zhu, Mingchao Feng, Meng Chen, Xiaodong He

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI、eess.AS

Comments Accepted by ICASSP 2022

Towards Learning Cross-Modal Perception-Trace Models

Achim Rettinger, Viktoria Bogdanova, Philipp Niemann

专题命中 跨模态检索 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Weakly-paired Cross-Modal Hashing

Xuanwu Liu, Jun Wang, Guoxian Yu, Carlotta Domeniconi, Xiangliang Zhang

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)

Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms

Avikalp Srivastava, Hsin Wen Liu, Sumio Fujita

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Submitted for review at CIKM 2019

Learning Decorrelated Hashing Codes for Multimodal Retrieval

Dayong Tian

专题命中 跨模态检索 ：multimodal(title,abstract)；image-text(abstract)

Ranking-based Deep Cross-modal Hashing

Xuanwu Liu, Guoxian Yu, Carlotta Domeniconi, Jun Wang, Yazhou Ren, Maozu Guo

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)

Joint Cluster Unary Loss for Efficient Cross-Modal Hashing

Shifeng Zhang, Jianmin Li, Bo Zhang

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)

Investigating non-classical correlations between decision fused multi-modal documents

Dimitris Gkoumas, Sagar Uprety, Dawei Song

专题命中 跨模态检索 ：multi-modal(title,abstract)；multimodal(abstract)

Cross-Modal Retrieval with Implicit Concept Association

Yale Song, Mohammad Soleymani

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering

Pan Lu, Hongsheng Li, Wei Zhang, Jianyong Wang, Xiaogang Wang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments To appear in AAAI 2018

Modality-specific Cross-modal Similarity Measurement with Recurrent Attention Network

Yuxin Peng, Jinwei Qi, Yuxin Yuan

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 13 pages, submitted to IEEE Transactions on Image Processing

Simple to Complex Cross-modal Learning to Rank

Minnan Luo, Xiaojun Chang, Zhihui Li, Liqiang Nie, Alexander G. Hauptmann, Qinghua Zheng

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)

Comments 14 pages; Accepted by Computer Vision and Image Understanding

Composite Correlation Quantization for Efficient Multimodal Retrieval

Mingsheng Long, Yue Cao, Jianmin Wang, Philip S. Yu

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)

Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

超越上下文的规模：文档理解的多模态检索增强生成综述

Sensen Gao, Shanshan Zhao, Xu Jiang, Lunhao Duan, Yong Xien Chng, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, Jia-Wang Bian, Mingming Gong

机构 * MBZUAI ； Alibaba Group（阿里巴巴集团） ； Tsinghua University（清华大学） ； Wuhan University（武汉大学） ； Nanyang Technological University（南洋理工大学） ； University of Melbourne（墨尔本大学） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

AI总结 本文综述了多模态检索增强生成在文档理解中的应用，提出基于领域、检索模态和粒度的分类体系，总结了关键数据集、基准测试和行业应用，指出了效率、细粒度表示和鲁棒性等挑战。

Comments Accepted by ACL2026 Main Conference; Project is available at https://github.com/SensenGao/Multimodal-RAG-Survey-For-Document

FUSE : Failure-aware Usage of Subagent Evidence for MultiModal Search and Recommendation

FUSE : 多模态搜索与推荐中子代理证据的故障感知使用

Tushar Vatsa, Vibha Belavadi, Priya Shanmugasundaram, Suhas Suresha, Dewang Sultania

机构 * Adobe Inc.（Adobe公司） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

AI总结 FUSE通过上下文压缩策略提升多模态搜索与推荐性能，实现93.3%的意图准确率和99.4%的召回率。

Comments ICDM MMSR 2025: Workshop on Multimodal Search and Recommendations

Towards Unbiased Cross-Modal Representation Learning for Food Image-to-Recipe Retrieval

迈向食品图像到食谱检索的无偏跨模态表示学习

Qing Wang, Chong-Wah Ngo, Ee-Peng Lim

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

AI总结 本文通过因果理论解决食品图像到食谱检索中的偏见问题，提出因果干预方法和多标签分类器，提升检索性能。

Comments Code link: https://github.com/GZWQ/Towards-Unbiased-Cross-Modal-Representation-Learning-for-Food-Image-to-Recipe-Retrieval

Smart Routing for Multimodal Video Retrieval: When to Search What

Kevin Dela Rosa

机构 * Cloudglue 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICCV 2025 Multimodal Representation and Retrieval Workshop

Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusion

Peiyuan Chen, Zecheng Zhang, Yiping Dong, Li Zhou, Han Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Visual Question Answering, Rank VQA, Faster R-CNN, BERT, Multimodal Fusion, Ranking Learning, Hybrid Training Strategy

Graph Pattern Loss based Diversified Attention Network for Cross-Modal Retrieval

Xueying Chen, Rong Zhang, Yibing Zhan

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 3 figures

Journal ref [1] Chen X , Zhang R , Zhan Y . Graph Pattern Loss Based Diversified Attention Network For Cross-Modal Retrieval[C]// 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020