多模态大模型

Multi-Modal Chorus Recognition for Improving Song Search

Jiaan Wang, Zhixu Li, Binbin Gu, Tingyi Zhang, Qingsheng Liu, Zhigang Chen

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments Accepted at the 30th International Conference on Artificial Neural Networks (ICANN 2021)

Listen to Your Favorite Melodies with img2Mxml, Producing MusicXML from Sheet Music Image by Measure-based Multimodal Deep Learning-driven Assembly

Tomoyuki Shishido, Fehmiju Fati, Daisuke Tokushige, Yasuhiro Ono

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 19 pages, 7 figures

Watching Too Much Television is Good: Self-Supervised Audio-Visual Representation Learning from Movies and TV Shows

Mahdi M. Kalayeh, Nagendra Kamath, Lingyi Liu, Ashok Chandrashekar

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Deep Learning Frameworks Applied For Audio-Visual Scene Classification

Lam Pham, Alexander Schindler, Mina Schütz, Jasmin Lampert, Sven Schlarb, Ross King

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 6 pages

Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning

Shaobo Min, Qi Dai, Hongtao Xie, Chuang Gan, Yongdong Zhang, Jingdong Wang

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV

S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations

Jheng-hao Lin, Yist Y. Lin, Chung-Ming Chien, Hung-yi Lee

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments Accepted by INTERSPEECH 2021

EMA2S: An End-to-End Multimodal Articulatory-to-Speech System

Yu-Wen Chen, Kuo-Hsuan Hung, Shang-Yi Chuang, Jonathan Sherman, Wen-Chin Huang, Xugang Lu, Yu Tsao

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Neural dSCA: demixing multimodal interaction among brain areas during naturalistic experiments

Yu Takagi, Laurence T. Hunt, Ryu Ohata, Hiroshi Imamizu, Jun-ichiro Hirayama

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Multi-modal Point-of-Care Diagnostics for COVID-19 Based On Acoustics and Symptoms

Srikanth Raj Chetupalli, Prashant Krishnan, Neeraj Sharma, Ananya Muguli, Rohit Kumar, Viral Nanda, Lancelot Mark Pinto, Prasanta Kumar Ghosh, Sriram Ganapathy

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments The Manuscript is submitted to IEEE-EMBS Journal of Biomedical and Health Informatics on June 1, 2021

Multi-Modal Semantic Inconsistency Detection in Social Media News Posts

Scott McCrae, Kehan Wang, Avideh Zakhor

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Exploiting Temporal Dependencies for Cross-Modal Music Piece Identification

Luis Carvalho, Gerhard Widmer

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 eess.AS

Comments 5 pages, 3 figures

Journal ref Proceedings of the 29th European Signal Processing Conference (EUSIPCO 2021), Dublin, Ireland

Recent Advances and Trends in Multimodal Deep Learning: A Review

Jabeen Summaira, Xi Li, Amin Muhammad Shoib, Songyuan Li, Jabbar Abdul

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multi-target DoA Estimation with an Audio-visual Fusion Mechanism

Xinyuan Qian, Maulik Madhavi, Zexu Pan, Jiadong Wang, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments ICASSP 2021 accepted

FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention

Yist Y. Lin, Chung-Ming Chien, Jheng-Hao Lin, Hung-yi Lee, Lin-shan Lee

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments To appear in the proceedings of ICASSP 2021, equal contribution from first two authors

Cross-Modal Music-Video Recommendation: A Study of Design Choices

Laure Pretet, Gael Richard, Geoffroy Peeters

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.MM

Multimodal Self-Supervised Learning of General Audio Representations

Luyu Wang, Pauline Luc, Adria Recasens, Jean-Baptiste Alayrac, Aaron van den Oord

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings

Utkarsh Sarawgi, Rishab Khincha, Wazeer Zulfikar, Satrajit Ghosh, Pattie Maes

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted at IJCNN 2021, to appear in IEEE proceedings. Equal contributions from US, RK and WZ

Fusing information streams in end-to-end audio-visual speech recognition

Wentao Yu, Steffen Zeiler, Dorothea Kolossa

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 5 pages

Journal ref Published in International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021

ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding

Minjeong Kim, Gyuwan Kim, Sang-Woo Lee, Jung-Woo Ha

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL

Comments ICASSP 2021; 5 pages

Self-Supervised learning with cross-modal transformers for emotion recognition

Aparna Khare, Srinivas Parthasarathy, Shiva Sundaram

专题命中 音频语音多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CL

Comments To appear in SLT2020

Interpretable Multi-Modal Hate Speech Detection

Prashanth Vijayaraghavan, Hugo Larochelle, Deb Roy

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL

Comments 5 pages, Accepted at the International Conference on Machine Learning AI for Social Good Workshop, Long Beach, United States, 2019

Journal ref ICML Workshop on AI for Social Good, 2019

Audio-Visual Target Speaker Enhancement on Multi-Talker Environment using Event-Driven Cameras

Ander Arriandiaga, Giovanni Morrone, Luca Pasa, Leonardo Badino, Chiara Bartolozzi

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted at ISCAS 2021

A Curated Dataset of Urban Scenes for Audio-Visual Scene Analysis

Shanshan Wang, Annamaria Mesaros, Toni Heittola, Tuomas Virtanen

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments accepted by ICASSP 2021

Audio-Visual Speech Inpainting with Deep Learning

Giovanni Morrone, Daniel Michelsanti, Zheng-Hua Tan, Jesper Jensen

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted at ICASSP 2021

Audio-Visual Floorplan Reconstruction

Senthil Purushwalkam, Sebastian Vicenc Amengual Gari, Vamsi Krishna Ithapu, Carl Schissler, Philip Robinson, Abhinav Gupta, Kristen Grauman

专题命中 音频语音多模态 ：audio-visual(title)；multi-modal(abstract)；分类 cs.CV

Detecting Hate Speech in Multi-modal Memes

Abhishek Das, Japsimar Singh Wahi, Siyao Li

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Temporal Bilinear Encoding Network of Audio-Visual Features at Low Sampling Rates

Feiyan Hu, Eva Mohedano, Noel O'Connor, Kevin McGuinness

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments 8 pages

Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoders

Mostafa Sadeghi, Simon Leglaive, Xavier Alameda-PIneda, Laurent Girin, Radu Horaud

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Submitted to IEEE/ACM Transactions on Audio, Speech, and Language Processing

Journal ref IEEE/ACM Transactions on Audio, Speech and Language Processing, 28, 2020

AV Taris: Online Audio-Visual Speech Recognition

George Sterpu, Naomi Harte

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Top-1 CORSMAL Challenge 2020 Submission: Filling Mass Estimation Using Multi-modal Observations of Human-robot Handovers

Vladimir Iashin, Francesca Palermo, Gökhan Solak, Claudio Coppola

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Code: https://github.com/v-iashin/CORSMAL Docker: https://hub.docker.com/r/iashin/corsmal