多模态大模型 - arXivDaily 专题

2108.04049 2021-10-20 cs.CL cs.IR 79%

Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models

Bogdan Kostić, Julian Risch, Timo Möller

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Comments Accepted at the 3rd Workshop on Machine Reading for Question Answering (MRQA) at EMNLP 2021

2110.04599 2021-10-12 cs.LG cs.MM 79%

Embed Everything: A Method for Efficiently Co-Embedding Multi-Modal Spaces

Sarah Di, Robin Yu, Amol Kapoor

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.MM

Comments 7 pages, 4 figures

2109.15153 2021-10-01 cs.CL 79%

Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism

Xiaoqiang Zhang, Ying Chen, Guangyuan Li

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Comments NLPCC 2021

2109.01850 2021-09-13 cs.CL 79%

Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

Wenjia Zhang, Lin Gui, Yulan He

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by the CIKM 2021

2109.00796 2021-09-06 cs.CV 79%

Multi-Modal Zero-Shot Sign Language Recognition

Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, Mohammad Sabokrou

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2108.10059

2108.03788 2021-08-19 cs.CV cs.IR 79%

Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images

Zhongwei Xie, Ling Liu, Lin Li, Luo Zhong

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by CIKM 2021. arXiv admin note: substantial text overlap with arXiv:2108.00705

2102.08655 2021-08-11 cs.CL 79%

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Nora Hollenstein, Cedric Renggli, Benjamin Glaus, Maria Barrett, Marius Troendle, Nicolas Langer, Ce Zhang

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Journal ref Frontiers of Human Neuroscience 2021

1911.12763 2021-07-14 cs.CV 79%

Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA

Mikhail Fain, Niall Twomey, Andrey Ponikar, Ryan Fox, Danushka Bollegala

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2107.02575 2021-07-07 cs.CV 79%

Contrastive Multimodal Fusion with TupleInfoNCE

Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2105.07921 2021-06-18 cs.CV 79%

BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval

Gencer Sumbul, Arne de Wall, Tristan Kreuziger, Filipe Marcelino, Hugo Costa, Pedro Benevides, Mário Caetano, Begüm Demir, Volker Markl

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at the IEEE Geoscience and Remote Sensing Magazine. Our code is available online at https://git.tu-berlin.de/rsim/BigEarthNet-MM_19-classes_models. arXiv admin note: substantial text overlap with arXiv:2001.06372

2101.05068 2021-06-15 cs.CV 79%

Probabilistic Embeddings for Cross-Modal Retrieval

Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio de Rezende, Yannis Kalantidis, Diane Larlus

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2021; Code is available at https://github.com/naver-ai/pcme

2105.07128 2021-05-18 cs.CV 79%

FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval

Xin Liu, Xingzhi Wang, Yiu-ming Cheung

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 7 figures

Journal ref IEEE Transactions on Neural Networks and Learning Systems, 2021

2105.03567 2021-05-11 cs.LG cs.AI cs.CR 79%

Multimodal and Contrastive Learning for Click Fraud Detection

Weibin Li, Qiwei Zhong, Qingyang Zhao, Hongchun Zhang, Xiaonan Meng

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments Accepted to DeMal@WWW 2021

2104.07098 2021-04-16 cs.CV 79%

StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis

Moustafa Meshry, Yixuan Ren, Larry S Davis, Abhinav Shrivastava

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

2104.04991 2021-04-13 cs.CV 79%

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Wei Chen, Yu Liu, Erwin M. Bakker, Michael S. Lew

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by Pattern Recognition

2103.13061 2021-03-25 cs.CV 79%

Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning

Amaia Salvador, Erhan Gundogdu, Loris Bazzani, Michael Donoser

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments CVPR 2021

2103.04731 2021-03-09 cs.CV 79%

Self-Augmented Multi-Modal Feature Embedding

Shinnosuke Matsuo, Seiichi Uchida, Brian Kenji Iwana

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at ICASSP2021

2006.06325 2021-01-14 cs.CV cs.LG eess.IV 79%

CoMIR: Contrastive Multimodal Image Representation for Registration

Nicolas Pielawski, Elisabeth Wetzer, Johan Öfverstedt, Jiahao Lu, Carolina Wählby, Joakim Lindblad, Nataša Sladoje

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 21 pages, 11 figures

Journal ref NeurIPS 2020

2101.02358 2021-01-08 cs.LG cs.CV stat.ML 79%

OAAE: Adversarial Autoencoders for Novelty Detection in Multi-modal Normality Case via Orthogonalized Latent Space

Sungkwon An, Jeonghoon Kim, Myungjoo Kang, Shahbaz Razaei, Xin Liu

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to AAAI 2021 Workshop: Towards Robust, Secure and Efficient Machine Learning

2012.13538 2020-12-29 cs.IR cs.CV 79%

Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing

Jun Yu, Hao Zhou, Yibing Zhan, Dacheng Tao

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2012.04329 2020-12-09 cs.CV 79%

StacMR: Scene-Text Aware Cross-Modal Retrieval

Andrés Mafla, Rafael Sampaio de Rezende, Lluís Gómez, Diane Larlus, Dimosthenis Karatzas

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2012.00682 2020-12-02 cs.LG cs.CV 79%

Learning Disentangled Latent Factors from Paired Data in Cross-Modal Retrieval: An Implicit Identifiable VAE Approach

Minyoung Kim, Ricardo Guerrero, Vladimir Pavlovic

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2011.12349 2020-12-01 cs.LG cs.CL cs.IR 79%

Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning

Batuhan Bardak, Mehmet Tan

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments 21 pages, 2 figures, Submitted to Elsevier (Artificial Intelligence in Medicine)

1910.03291 2020-11-02 cs.CL cs.IR cs.LG 79%

Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task

Alireza Mohammadshahi, Remi Lebret, Karl Aberer

专题命中跨模态检索：cross-modal(title)；multimodal(abstract)；分类 cs.CL

2010.04339 2020-10-12 cs.CV cs.RO 79%

MMGSD: Multi-Modal Gaussian Shape Descriptors for Correspondence Matching in 1D and 2D Deformable Objects

Aditya Ganapathi, Priya Sundaresan, Brijen Thananjeyan, Ashwin Balakrishna, Daniel Seita, Ryan Hoque, Joseph E. Gonzalez, Ken Goldberg

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments IROS 2020 Workshop on Managing Deformation: A Step Towards Higher Robot Autonomy

2002.02698 2020-10-07 cs.CV 79%

Deep Robust Multilevel Semantic Cross-Modal Hashing

Ge Song, Jun Zhao, Xiaoyang Tan

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 9 figures, submitted to a journal

2007.06390 2020-10-02 cs.CL cs.IR cs.LG 79%

A Feature Analysis for Multimodal News Retrieval

Golsa Tahmasebzadeh, Sherzod Hakimov, Eric Müller-Budack, Ralph Ewerth

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments CLEOPATRA Workshop co-located with ESWC 2020

Journal ref CLEOPATRA Workshop co-located with ESWC 2020

2009.09809 2020-09-22 cs.CV 79%

Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval

Andres Mafla, Sounak Dey, Ali Furkan Biten, Lluis Gomez, Dimosthenis Karatzas

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2009.07162 2020-09-16 cs.CL 79%

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product

Tiangang Zhu, Yue Wang, Haoran Li, Youzheng Wu, Xiaodong He, Bowen Zhou

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2020

2007.15103 2020-08-12 cs.CV cs.IR 79%

Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval

Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted for ORAL presentation in BMVC 2020

1. 跨模态检索 3450 篇

Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models

Embed Everything: A Method for Efficiently Co-Embedding Multi-Modal Spaces

Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism

Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

Multi-Modal Zero-Shot Sign Language Recognition

Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA

Contrastive Multimodal Fusion with TupleInfoNCE

BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval

Probabilistic Embeddings for Cross-Modal Retrieval

FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval

Multimodal and Contrastive Learning for Click Fraud Detection

StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning

Self-Augmented Multi-Modal Feature Embedding

CoMIR: Contrastive Multimodal Image Representation for Registration

OAAE: Adversarial Autoencoders for Novelty Detection in Multi-modal Normality Case via Orthogonalized Latent Space

Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing

StacMR: Scene-Text Aware Cross-Modal Retrieval

Learning Disentangled Latent Factors from Paired Data in Cross-Modal Retrieval: An Implicit Identifiable VAE Approach

Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning

Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task

MMGSD: Multi-Modal Gaussian Shape Descriptors for Correspondence Matching in 1D and 2D Deformable Objects

Deep Robust Multilevel Semantic Cross-Modal Hashing

A Feature Analysis for Multimodal News Retrieval

Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product

Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval