多模态大模型

Exploring Multimodal Sentiment Analysis via CBAM Attention and Double-layer BiLSTM Architecture

Huiru Wang, Xiuhong Li, Zenyu Ren, Dan Yang, chunming Ma

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Vision, Deduction and Alignment: An Empirical Study on Multi-modal Knowledge Graph Alignment

Yangning Li, Jiaoyan Chen, Yinghui Li, Yuejia Xiang, Xi Chen, Hai-Tao Zheng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted by ICASSP2023

Using Auxiliary Tasks In Multimodal Fusion Of Wav2vec 2.0 And BERT For Multimodal Emotion Recognition

Dekai Sun, Yancheng He, Jiqing Han

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review

Can Cui, Haichun Yang, Yaohong Wang, Shilin Zhao, Zuhayr Asad, Lori A. Coburn, Keith T. Wilson, Bennett A. Landman, Yuankai Huo

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Learning Sequential Latent Variable Models from Multimodal Time Series Data

Oliver Limoyo, Trevor Ablett, Jonathan Kelly

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments In: Petrovic, I., Menegatti, E., Marković, I. (eds) Intelligent Autonomous Systems 17. IAS 2022. Lecture Notes in Networks and Systems, vol 577. Springer, Cham

Multimodal Sequential Generative Models for Semi-Supervised Language Instruction Following

Kei Akuzawa, Yusuke Iwasawa, Yutaka Matsuo

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Survey on Self-Supervised Multimodal Representation Learning and Foundation Models

Sushil Thapa

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Data Augmentation for Visual-Infrared Person ReID with Corrupted Data

Arthur Josi, Mahdi Alehdaghi, Rafael M. O. Cruz, Eric Granger

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 8 pages of main content, 2 pages of references, 2 pages of supplementary material, 3 figures, WACV 2023 RWS workshop,

MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion

Yu Zhao, Xiangrui Cai, Yike Wu, Haiwei Zhang, Ying Zhang, Guoqing Zhao, Ning Jiang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by EMNLP 2022

MMGA: Multimodal Learning with Graph Alignment

Xuan Yang, Quanjin Tao, Xiao Feng, Donghong Cai, Xiang Ren, Yang Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments Please contact xuany@zju.edu.cn for the dataset

A Multimodal Sensor Fusion Framework Robust to Missing Modalities for Person Recognition

Vijay John, Yasutomo Kawanishi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted for ACM Multimedia Asia, 2022

A survey on Self Supervised learning approaches for improving Multimodal representation learning

Naman Goyal

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Transfer Learning with Joint Fine-Tuning for Multimodal Sentiment Analysis

Guilherme Lourenço de Toledo, Ricardo Marcondes Marcacini

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Talk: https://icml.cc/Conferences/2022/ScheduleMultitrack?event=13483#collapse20429

Multi-Modal Fusion Transformer for Visual Question Answering in Remote Sensing

Tim Siebert, Kai Norman Clasen, Mahdyar Ravanbakhsh, Begüm Demir

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted in SPIE Remote Sensing (ESI22R)

Scalable Self-Supervised Representation Learning from Spatiotemporal Motion Trajectories for Multimodal Computer Vision

Swetava Ganguli, C. V. Krishnakumar Iyer, Vipul Pandey

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Extended abstract accepted for presentation at BayLearn 2022. 3 pages, 2 figures, 1 table. Abstract based on IEEE MDM 2022 research track paper: arXiv:2110.12521

Cross-Modality Gated Attention Fusion for Multimodal Sentiment Analysis

Ming Jiang, Shaoxiong Ji

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Reachability Embeddings: Scalable Self-Supervised Representation Learning from Mobility Trajectories for Multimodal Geospatial Computer Vision

Swetava Ganguli, C. V. Krishnakumar Iyer, Vipul Pandey

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Extended version of the accepted research track paper at the 23rd IEEE International Conference on Mobile Data Management (MDM), 2022, Paphos, Cyprus. 12 pages, 6 figures, 3 tables

Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognition

Zihan Zhao, Yanfeng Wang, Yu Wang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to INTERSPEECH 2022

Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness

Paul Pu Liang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Annual Meeting of the Association for the Scientific Study of Consciousness 2022

Multimodal Knowledge Alignment with Reinforcement Learning

Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, JaeSung Park, Ximing Lu, Prithviraj Ammanabrolu, Rowan Zellers, Ronan Le Bras, Gunhee Kim, Yejin Choi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Semi-Supervised Learning for Text Recognition

Aviad Aberdam, Roy Ganz, Shai Mazor, Ron Litman

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Code will be published upon publication

Multimodal Representation Learning With Text and Images

Aishwarya Jayagopal, Ankireddy Monica Aiswarya, Ankita Garg, Srinivasan Kolumam Nandakumar

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding

Alina Roitberg, Kunyu Peng, Zdravko Marinov, Constantin Seibold, David Schneider, Rainer Stiefelhagen

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at Intelligent Vehicles Symposium 2022, IEEE

Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization

Zhengqi Gao, Sucheng Ren, Zihui Xue, Siting Li, Hang Zhao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、eess.AS

M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining

Xiao Dong, Xunlin Zhan, Yangxin Wu, Yunchao Wei, Michael C. Kampffmeyer, Xiaoyong Wei, Minlong Lu, Yaowei Wang, Xiaodan Liang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments CVPR2022

Sememe Prediction for BabelNet Synsets using Multilingual and Multimodal Information

Fanchao Qi, Chuancheng Lv, Zhiyuan Liu, Xiaojun Meng, Maosong Sun, Hai-Tao Zheng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by Findings of ACL 2022 as a long paper. Camera-ready version

Multi-modal Graph Learning for Disease Prediction

Shuai Zheng, Zhenfeng Zhu, Zhizhe Liu, Zhenyu Guo, Yang Liu, Yuchen Yang, Yao Zhao

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Published by IEEE TMI

A Multi-modal Fusion Framework Based on Multi-task Correlation Learning for Cancer Prognosis Prediction

Kaiwen Tan, Weixian Huang, Xiaofeng Liu, Jinlong Hu, Shoubin Dong

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition

Pengfei Liu, Kun Li, Helen Meng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Published in INTERSPEECH-2020

Journal ref INTERSPEECH 2020

MoCA: Incorporating Multi-stage Domain Pretraining and Cross-guided Multimodal Attention for Textbook Question Answering

Fangzhi Xu, Qika Lin, Jun Liu, Lingling Zhang, Tianzhe Zhao, Qi Chai, Yudai Pan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments 9 pages, 6 figures