多模态大模型

ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology

Srikumar Sastry, Subash Khanal, Aayush Dhakal, Jiayu Lin, Dan Cher, Phoenix Jarosz, Nathan Jacobs

机构 * Washington University in St. Louis（华盛顿大学圣路易斯分校） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；any-to-any(abstract)；分类 cs.CV

Comments 21 pages, 16 figures

SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model

Lin Lin, Jiefeng Long, Zhihe Wan, Yuchi Wang, Dingkang Yang, Shuang Yang, Yueyang Yao, Xu Chen, Zirui Guo, Shengqiang Li, Weiran Li, Hanyu Li, Yaling Mou, Yan Qiu, Haiyang Yu, Xiao Liang, Hongsheng Li, Chao Feng

机构 * ByteDance Douyin SAIL Team, CUHK MMLab（字节跳动抖音SAIL团队，CUHK MMLab） 纠错

专题命中 跨模态检索 ：omni-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Technical Report

Exploring a Unified Vision-Centric Contrastive Alternatives on Multi-Modal Web Documents

Yiqi Lin, Alex Jinpeng Wang, Linjie Li, Zhengyuan Yang, Mike Zheng Shou

机构 * Show Lab, National University of Singapore（新加坡国立大学展示实验室） ； Central South University（中南大学） ； Microsoft（微软公司） 纠错

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；cross-modal(abstract)；image-text(abstract)

Comments Project page: this https://linyq17.github.io/VC2L/

Generalized Contrastive Learning for Universal Multimodal Retrieval

Jungsoo Lee, Janghoon Cho, Hyojin Park, Munawar Hayat, Kyuwoong Hwang, Fatih Porikli, Sungha Choi

机构 * Qualcomm AI Research（高通人工智能研究） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2025

CPCL: Cross-Modal Prototypical Contrastive Learning for Weakly Supervised Text-based Person Retrieval

Xinpeng Zhao, Yanwei Zheng, Chuanlin Lan, Xiaowei Zhang, Bowen Huang, Jibin Yang, Dongxiao Yu

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 9 pages, 6 figures, under peer review

Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution

Junyi Yuan, Jian Zhang, Fangyu Wu, Dongming Lu, Huanda Lu, Qiufeng Wang

机构 * Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学） ； NingboTech University（宁波科技大学） ； Zhejiang University（浙江大学） 纠错

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Anatomy-Aware Conditional Image-Text Retrieval

Meng Zheng, Jiajin Zhang, Benjamin Planche, Zhongpai Gao, Terrence Chen, Ziyan Wu

专题命中 跨模态检索 ：image-text(title,abstract)；multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments 16 pages, 10 figures

Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval

Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2025

Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey

Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, Yong Liu, Jing Shao, Hui Xiong, Xuming Hu

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CL

Contrastive Knowledge Distillation for Robust Multimodal Sentiment Analysis

Zhongyi Sang, Kotaro Funakoshi, Manabu Okumura

专题命中 跨模态检索 ：multimodal(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.MM

Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications

David Restrepo, Chenwei Wu, Sebastián Andrés Cajas, Luis Filipe Nakayama, Leo Anthony Celi, Diego M López

专题命中 跨模态检索 ：multimodal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.AI

Multi-Modal Generative Embedding Model

Feipeng Ma, Hongwei Xue, Guangting Wang, Yizhou Zhou, Fengyun Rao, Shilin Yan, Yueyi Zhang, Siying Wu, Mike Zheng Shou, Xiaoyan Sun

专题命中 跨模态检索 ：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Hallucination Augmented Contrastive Learning for Multimodal Large Language Model

Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, Shikun Zhang

专题命中 跨模态检索 ：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；cross-modal(abstract)

Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning

Md Mahedi Hasan, Shoaib Meraj Sami, Nasser Nasrabadi

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

Connecting Multi-modal Contrastive Representations

Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Li Tang, Linjun Li, Yongqi Wang, Aoxiong Yin, Ziang Zhang, Zhou Zhao

专题命中 跨模态检索 ：multi-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments NeurIPS 2023

AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning

Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, Hai Jin

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments This paper has been accepted by the ACM International Conference on Multimedia (ACM MM '23, October 29-November 3, 2023, Ottawa, ON, Canada)

Using Multiple Instance Learning to Build Multimodal Representations

Peiqi Wang, William M. Wells, Seth Berkowitz, Steven Horng, Polina Golland

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training

Chulun Zhou, Yunlong Liang, Fandong Meng, Jinan Xu, Jinsong Su, Jie Zhou

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CL

Comments accepted to ACL 2023 Findings

Multimodal Federated Learning via Contrastive Representation Ensemble

Qiying Yu, Yang Liu, Yimu Wang, Ke Xu, Jingjing Liu

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.AI

Comments ICLR 2023, update

Learnable Pillar-based Re-ranking for Image-Text Retrieval

Leigang Qu, Meng Liu, Wenjie Wang, Zhedong Zheng, Liqiang Nie, Tat-Seng Chua

专题命中 跨模态检索 ：image-text(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by SIGIR'2023

Journal ref Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023)

Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval

Ding Jiang, Mang Ye

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023. Codes are available at this https://github.com/anosorae/IRRA

ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval

Nicola Messina, Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Fabrizio Falchi, Giuseppe Amato, Rita Cucchiara

专题命中 跨模态检索 ：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CBMI 2022

UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning

Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, Haifeng Wang

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CL

Comments The paper has been accepted by the main conference of ACL2021 as a long paper

Towards Efficient Cross-Modal Visual Textual Retrieval using Transformer-Encoder Deep Features

Nicola Messina, Giuseppe Amato, Fabrizio Falchi, Claudio Gennaro, Stéphane Marchand-Maillet

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at CBMI 2021

Learning Multi-Modal Nonlinear Embeddings: Performance Bounds and an Algorithm

Semih Kaya, Elif Vural

专题命中 跨模态检索 ：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Continual learning in cross-modal retrieval

Kai Wang, Luis Herranz, Joost van de Weijer

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 2nd CLVISION workshop in CVPR 2021

CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval

Hai X. Pham, Ricardo Guerrero, Jiatong Li, Vladimir Pavlovic

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 22 pages, accepted in AAAI 2021

Matching Images and Text with Multi-modal Tensor Fusion and Re-ranking

Tan Wang, Xing Xu, Yang Yang, Alan Hanjalic, Heng Tao Shen, Jingkuan Song

专题命中 跨模态检索 ：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ACM Multimedia 2019 Oral

Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models

Jiuxiang Gu, Jianfei Cai, Shafiq Joty, Li Niu, Gang Wang

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 10 pages, 6 figures, Accepted as spotlight at CVPR 2018

Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification

先定位再排序：基于知识的视觉问答中无训练实体识别的再探讨

Qian Ma, Qiong Wu, Zhengyi Zhou, Yao Ma

机构 * Rensselaer Polytechnic Institute（伦斯勒理工学院） ； AT&T Chief Data Office（AT&T首席数据办公室） 纠错

专题命中 跨模态检索 ：MLLM(summary_cn,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

AI总结 针对知识型视觉问答中实体与证据双重定位瓶颈，提出解耦实体识别与段落排序的无训练IBA框架，利用MLLM候选名称选择与文本重排序器，在降低复杂度同时提升性能。

Comments Accepted by ACL 2026 Findings. Project page https://github.com/VAN-QIAN/ACL26-IBA/