多模态大模型 - arXivDaily 专题

2112.01368 2021-12-03 cs.CL cs.AI cs.LG 81%

ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors

Huaishao Luo, Lei Ji, Yanyong Huang, Bin Wang, Shenggong Ji, Tianrui Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

2108.02923 2021-11-09 cs.CV cs.CL 81%

StrucTexT: Structured Text Understanding with Multi-Modal Transformers

Yulin Li, Yuxi Qian, Yuchen Yu, Xiameng Qin, Chengquan Zhang, Yan Liu, Kun Yao, Junyu Han, Jingtuo Liu, Errui Ding

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ACM Multimedia 2021. 9 pages

2106.13033 2021-10-27 cs.CV cs.CL 81%

A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021

Ke-Han Lu, Bo-Han Fang, Kuan-Yu Chen

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments CVPR 2021 Workshop: Visual Question Answering (VQA) Challenge

2104.12763 2021-10-13 cs.CV cs.CL cs.LG 81%

MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding

Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, Nicolas Carion

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL

2102.02340 2021-10-07 cs.LG cs.AI cs.CL 81%

MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records

Zhen Xu, David R. So, Andrew M. Dai

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication at the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21)

2110.00416 2021-10-04 cs.MM cs.CL cs.LG 81%

FiLMing Multimodal Sarcasm Detection with Attention

Sundesh Gupta, Aditya Shah, Miten Shah, Laribok Syiemlieh, Chandresh Maurya

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments 13 pages

2109.12724 2021-09-28 cs.CV cs.AI 81%

Research on facial expression recognition based on Multimodal data fusion and neural network

Yi Han, Xubin Wang, Zhengyu Lu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

2109.00412 2021-09-17 cs.CL cs.AI 81%

Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis

Wei Han, Hui Chen, Soujanya Poria

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a long paper at EMNLP 2021

2001.06888 2021-09-16 cs.CL cs.LG cs.MM cs.SI 81%

A multimodal deep learning approach for named entity recognition from social media

Meysam Asgari-Chenaghlu, M. Reza Feizi-Derakhshi, Leili Farzinvash, M. A. Balafar, Cina Motamed

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.MM

2109.00922 2021-09-10 cs.LG cs.AI cs.CL 81%

Improving Multimodal fusion via Mutual Dependency Maximisation

Pierre Colombo, Emile Chapuis, Matthieu Labeau, Chloe Clavel

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref EMNLP 2021

2102.10772 2021-08-19 cs.CV cs.CL 81%

UniT: Multimodal Multitask Learning with a Unified Transformer

Ronghang Hu, Amanpreet Singh

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 16 pages

2107.06779 2021-07-15 cs.CL cs.SD eess.AS 81%

MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation

Jingwen Hu, Yuchen Liu, Jinming Zhao, Qin Jin

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、eess.AS

2107.00648 2021-07-02 cs.CV cs.LG cs.MM q-bio.GN q-bio.QM 81%

Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Data

Nathaniel Braman, Jacob W. H. Gordon, Emery T. Goossens, Caleb Willis, Martin C. Stumpe, Jagadish Venkataraman

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted for presentation at MICCAI 2021

2106.14467 2021-06-29 cs.CV cs.AI 81%

Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning

Yi Zhang, Sheng Huang, Xi Peng, Dan Yang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages

2105.07878 2021-05-19 cs.AI cs.CV 81%

A Review on Explainability in Multimodal Deep Neural Nets

Gargi Joshi, Rahee Walambe, Ketan Kotecha

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 24 pages 6 figures

Journal ref in IEEE Access, vol. 9, pp. 59800-59821, 2021

2104.09224 2021-04-20 cs.CV cs.AI cs.LG cs.RO 81%

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Aditya Prakash, Kashyap Chitta, Andreas Geiger

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2021

2104.08405 2021-04-20 cs.CL cs.CV cs.IR 81%

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Te-Lin Wu, Cheng Li, Mingyang Zhang, Tao Chen, Spurthi Amba Hombaiah, Michael Bendersky

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

2103.07659 2021-03-16 cs.CL cs.MM 81%

Targeted aspect based multimodal sentiment analysis:an attention capsule extraction and multi-head fusion network

Jiaqian Wang, Donghong Gu, Chi Yang, Yun Xue, Zhengxin Song, Haoliang Zhao, Luwei Xiao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.MM

2103.00466 2021-03-02 cs.CV cs.CL 81%

NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes

Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 3rd rank DravidianLangTech workshop shared task, EACL-2021, 7 pages

2102.01998 2021-02-04 cs.AI cs.CV cs.IT cs.LG math.IT 81%

Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond

Guang Yang, Qinghao Ye, Jun Xia

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 68 pages, 19 figures, submitted to the Information Fusion journal

2011.04592 2020-11-10 cs.CL cs.CV 81%

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)

2010.11685 2020-10-23 cs.CV cs.AI 81%

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding

Zilong Wang, Mingjie Zhan, Xuebo Liu, Ding Liang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to EMNLP 2020 Findings

2010.08708 2020-10-20 cs.CV cs.CL 81%

Answer-checking in Context: A Multi-modal FullyAttention Network for Visual Question Answering

Hantao Huang, Tao Han, Wei Han, Deep Yap, Cheng-Ming Chiang

专题命中多模态训练与对齐：multi-modal(title)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted in ICPR2020

2010.05379 2020-10-13 cs.CL cs.CV cs.LG 81%

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding

Qinxin Wang, Hao Tan, Sheng Shen, Michael W. Mahoney, Zhewei Yao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

2010.01082 2020-10-05 cs.CL cs.AI 81%

Multi-Modal Open-Domain Dialogue

Kurt Shuster, Eric Michael Smith, Da Ju, Jason Weston

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL、cs.AI

2009.09926 2020-09-24 cs.CV cs.AI cs.IR cs.LG 81%

Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation

Po Li, Lei Li, Yan Fu, Jun Rong, Yu Zhang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.AI

2004.11838 2020-04-27 cs.CV cs.CY cs.LG cs.MM 81%

Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response

Ferda Ofli, Firoj Alam, Muhammad Imran

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted in ISCRAM 2020

1911.06258 2020-03-26 cs.CV cs.CL 81%

Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA

Ronghang Hu, Amanpreet Singh, Trevor Darrell, Marcus Rohrbach

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments CVPR 2020

2003.04981 2020-03-12 cs.CL cs.CV cs.LG stat.ML 81%

SAFE: Similarity-Aware Multi-Modal Fake News Detection

Xinyi Zhou, Jindi Wu, Reza Zafarani

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments To be published in The 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2020)

1910.04964 2020-01-07 cs.MM cs.CV cs.IR cs.LG 81%

Multi-modal Deep Analysis for Multimedia

Wenwu Zhu, Xin Wang, Hongzhi Li

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 25 pages, 39 figures, IEEE Transactions on Circuits and Systems for Video Technology

1. 多模态训练与对齐 6878 篇

ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors

StrucTexT: Structured Text Understanding with Multi-Modal Transformers

A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021

MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding

MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records

FiLMing Multimodal Sarcasm Detection with Attention

Research on facial expression recognition based on Multimodal data fusion and neural network

Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis

A multimodal deep learning approach for named entity recognition from social media

Improving Multimodal fusion via Mutual Dependency Maximisation

UniT: Multimodal Multitask Learning with a Unified Transformer

MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation

Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Data

Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning

A Review on Explainability in Multimodal Deep Neural Nets

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Targeted aspect based multimodal sentiment analysis:an attention capsule extraction and multi-head fusion network

NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes

Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding

Answer-checking in Context: A Multi-modal FullyAttention Network for Visual Question Answering

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding

Multi-Modal Open-Domain Dialogue

Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation

Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response

Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA

SAFE: Similarity-Aware Multi-Modal Fake News Detection

Multi-modal Deep Analysis for Multimedia