多模态大模型

ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis

Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by ACL Findings 2023

Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes

Chun Liu, Suqiang Ma, Zheng Li, Wei Yang, Zhigang Han

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

On the Generalization of Multi-modal Contrastive Learning

Qi Zhang, Yifei Wang, Yisen Wang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Lexical Retrieval Hypothesis in Multimodal Context

Po-Ya Angela Wang, Pin-Er Chen, Hsin-Yu Chou, Yu-Hsiang Tseng, Shu-Kai Hsieh

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval

Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Yanjun Wang

专题命中 跨模态检索 ：image-text(title,abstract)；分类 cs.CV

Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal

Zhitao Liu, Zengyu Liu, Jiwei Wei, Guan Wang, Zhenjiang Du, Ning Xie, Heng Tao Shen

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Denoising Multi-modal Sequential Recommenders with Contrastive Learning

Dong Yao, Shengyu Zhang, Zhou Zhao, Jieming Zhu, Wenqiao Zhang, Rui Zhang, Xiaofei He, Fei Wu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.MM

Deep Lifelong Cross-modal Hashing

Liming Xu, Hanqi Li, Bochuan Zheng, Weisheng Li, Jiancheng Lv

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

HyMo: Vulnerability Detection in Smart Contracts using a Novel Multi-Modal Hybrid Model

Mohammad Khodadadi, Jafar Tahmoresnezhad

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images

Xindi Wu, KwunFung Lau, Francesco Ferroni, Aljoša Ošep, Deva Ramanan

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments 12 pages, 8 figures

Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data

Paul Hager, Martin J. Menten, Daniel Rueckert

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted in CVPR 2023

DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-Based Segmentation Networks

Francesco Barbato, Giulia Rizzoli, Pietro Zanuttigh

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ICASSP 2023

MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representations

Ye Wang, Bowei Jiang, Changqing Zou, Rui Ma

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 14 figures

MRIS: A Multi-modal Retrieval Approach for Image Synthesis on Diverse Modalities

Boqi Chen, Marc Niethammer

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Cross-Modality Sub-Image Retrieval using Contrastive Multimodal Image Representations

Eva Breznik, Elisabeth Wetzer, Joakim Lindblad, Nataša Sladoje

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Semantic-Preserving Augmentation for Robust Image-Text Retrieval

Sunwoo Kim, Kyuhong Shim, Luong Trung Nguyen, Byonghyo Shim

专题命中 跨模态检索 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted to ICASSP 2023

X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation

Tom van Sonsbeek, Marcel Worring

专题命中 跨模态检索 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV

Comments IPMI 2023

Improving Visual-Semantic Embeddings by Learning Semantically-Enhanced Hard Negatives for Cross-modal Information Retrieval

Yan Gong, Georgina Cosma

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Journal ref Pattern Recognition 137 (2023): 109272

HADA: A Graph-based Amalgamation Framework in Image-text Retrieval

Manh-Duy Nguyen, Binh T. Nguyen, Cathal Gurrin

专题命中 跨模态检索 ：image-text(title,abstract)；分类 cs.CV

Transformer-based Cross-Modal Recipe Embeddings with Large Batch Training

Jing Yang, Junwen Chen, Keiji Yanai

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at MMM2023

SLAN: Self-Locator Aided Network for Cross-Modal Understanding

Jiang-Tian Zhai, Qi Zhang, Tong Wu, Xing-Yu Chen, Jiang-Jiang Liu, Bo Ren, Ming-Ming Cheng

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments 12 pages

Named Entity and Relation Extraction with Multi-Modal Retrieval

Xinyu Wang, Jiong Cai, Yong Jiang, Pengjun Xie, Kewei Tu, Wei Lu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2022. Code is publicly available at http://github.com/modelscope/adaseq/examples/MoRe

Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study

Jiahao Lu, Johan Öfverstedt, Joakim Lindblad, Nataša Sladoje

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments 37 pages, 10 figures, 2 tables

PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding

Honggu Zhou, Xiaogang Peng, Jiawei Mao, Zizhao Wu, Ming Zeng

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments In order to revise the paper

UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition

Guimin Hu, Ting-En Lin, Yi Zhao, Guangming Lu, Yuchuan Wu, Yongbin Li

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2022 main conference

Graph Neural Networks for Multimodal Single-Cell Data Integration

Hongzhi Wen, Jiayuan Ding, Wei Jin, Yiqi Wang, Yuying Xie, Jiliang Tang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by KDD 2022 Applied Data Science Track

Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval

Abhra Chaudhuri, Massimiliano Mancini, Yanbei Chen, Zeynep Akata, Anjan Dutta

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments British Machine Vision Conference (BMVC) 2022

Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval

Xuri Ge, Fuhai Chen, Songpei Xu, Fuxiang Tao, Joemon M. Jose

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments accepted to WACV 2023

ConTra: (Con)text (Tra)nsformer for Cross-Modal Video Retrieval

Adriano Fragomeni, Michael Wray, Dima Damen

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted in ACCV 2022

MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoning

Shangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin, Wei Chen, Lei Zhao

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI