多模态大模型

Efficient Multi-Modal Embeddings from Structured Data

Anita L. Verő, Ann Copestake

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments 5 pages, 5 pages of appendix, 7 figures

Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images and Recipes with Semantic Consistency and Attention Mechanism

Hao Wang, Doyen Sahoo, Chenghao Liu, Ke Shu, Palakorn Achananuparp, Ee-peng Lim, Steven C. H. Hoi

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments IEEE Transactions on Multimedia

SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation

Xin Wang, Yasheng Wang, Fei Mi, Pingyi Zhou, Yao Wan, Xiao Liu, Li Li, Hao Wu, Jin Liu, Xin Jiang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 3 figures, 5 tables

Telling the What while Pointing to the Where: Multimodal Queries for Image Retrieval

Soravit Changpinyo, Jordi Pont-Tuset, Vittorio Ferrari, Radu Soricut

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments IEEE/CVF International Conference on Computer Vision (ICCV 2021)

Efficient Deep Feature Calibration for Cross-Modal Joint Embedding Learning

Zhongwei Xie, Ling Liu, Lin Li, Luo Zhong

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments accepted by ACM ICMI 2021 conference

I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion

Ivona Tautkute, Tomasz Trzcinski

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Major edits

PatentNet: A Large-Scale Incomplete Multiview, Multimodal, Multilabel Industrial Goods Image Database

Fangyuan Lei, Da Huang, Jianjian Jiang, Ruijun Ma, Senhong Wang, Jiangzhong Cao, Yusen Lin, Qingyun Dai

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages,7 figures

Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embedding

Narjes Nikzad-Khasmakhi, Mohammad-Reza Feizi-Derakhshi, Meysam Asgari-Chenaghlu, Mohammad-Ali Balafar, Ali-Reza Feizi-Derakhshi, Taymaz Rahkar-Farshi, Majid Ramezani, Zoleikha Jahanbakhsh-Nagadeh, Elnaz Zafarani-Moattar, Mehrdad Ranjbar-Khadivi

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Deep Cross-modal Hashing via Margin-dynamic-softmax Loss

Rong-Cheng Tu, Xian-Ling Mao, Rongxin Tu, Binbin Bian, Wei Wei, Heyan Huang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training

Minheng Ni, Haoyang Huang, Lin Su, Edward Cui, Taroon Bharti, Lijuan Wang, Jianfeng Gao, Dongdong Zhang, Nan Duan

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2021

Memory Enhanced Embedding Learning for Cross-Modal Video-Text Retrieval

Rui Zhao, Kecheng Zheng, Zheng-Jun Zha, Hongtao Xie, Jiebo Luo

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Technical Report

Cross-modal Image Retrieval with Deep Mutual Information Maximization

Chunbin Gu, Jiajun Bu, Xixi Zhou, Chengwei Yao, Dongfang Ma, Zhi Yu, Xifeng Yan

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 35 pages,7 figures, Submitted to Neuralcomputing

Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers

Lisa Anne Hendricks, John Mellor, Rosalia Schneider, Jean-Baptiste Alayrac, Aida Nematzadeh

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments pre-print of MIT Press Publication version

Deep Hashing for Secure Multimodal Biometrics

Veeru Talreja, Matthew Valenti, Nasser Nasrabadi

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref IEEE Transactions on Information Forensics and Security,vol.16,pp.1306-1321,2021

Deep multi-modal networks for book genre classification based on its cover

Chandra Kundu, Lukun Zheng

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments 23 pages, 8 figures

Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning

Wanyun Cui, Guangyu Zheng, Wei Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Published at EMNLP 2020

MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model

Han Fu, Rui Wu, Chenghao Liu, Jianling Sun

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2020

Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Images

Hao Wang, Doyen Sahoo, Chenghao Liu, Ee-peng Lim, Steven C. H. Hoi

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted at CVPR 2019

Cluster-wise Unsupervised Hashing for Cross-Modal Similarity Search

Lu Wang, Jie Yang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 13 pages, 26 figures

MULE: Multimodal Universal Language Embedding

Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, Bryan A. Plummer

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted as an oral at AAAI 2020

Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations

Po-Yao Huang, Xiaojun Chang, Alexander Hauptmann

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at EMNLP 2019

Embedding Multimodal Relational Data for Knowledge Base Completion

Pouya Pezeshkpour, Liyan Chen, Sameer Singh

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Published at EMNLP 2018

Do Neural Network Cross-Modal Mappings Really Bridge Modalities?

Guillem Collell, Marie-Francine Moens

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments To appear at ACL 2018

Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings

Micael Carvalho, Rémi Cadène, David Picard, Laure Soulier, Nicolas Thome, Matthieu Cord

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments accepted at the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, 2018

MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval

Xin Huang, Yuxin Peng, Mingkuan Yuan

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 12 pages, submitted to IEEE Transactions on Cybernetics

Full-Network Embedding in a Multimodal Embedding Pipeline

Armand Vilalta, Dario Garcia-Gasulla, Ferran Parés, Eduard Ayguadé, Jesus Labarta, Ulises Cortés, Toyotaro Suzumura

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments In 2nd Workshop on Semantic Deep Learning (SemDeep-2) at the 12th International Conference on Computational Semantics (IWCS) 2017

Multi-modal image retrieval with random walk on multi-layer graphs

Renata Khasanova, Xiaowen Dong, Pascal Frossard

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.MM

Cross-Modal Scene Networks

Yusuf Aytar, Lluis Castrejon, Carl Vondrick, Hamed Pirsiavash, Antonio Torralba

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments See more at http://cmplaces.csail.mit.edu/. arXiv admin note: text overlap with arXiv:1607.07295

Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings

Spandana Gella, Mirella Lapata, Frank Keller

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 11 pages, NAACL-HLT 2016

Multimodal Convolutional Neural Networks for Matching Image and Sentence

Lin Ma, Zhengdong Lu, Lifeng Shang, Hang Li

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICCV 2015