多模态大模型

Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection

Jing Du, Shiliang Pu, Qinbo Dong, Chao Jin, Xin Qi, Dian Gu, Ru Wu, Hongwei Zhou

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.AI、eess.AS

Comments submit to ICASSP2022, 5 pages, 3 figures

Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization

Hao Jiang, Calvin Murdock, Vamsi Krishna Ithapu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

ASMD: an automatic framework for compiling multimodal datasets with audio and scores

Federico Simonetta, Stavros Ntalampiras, Federico Avanzini

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM、eess.AS

Comments Accepted at the Sound and Music Computing Conference 2020

Hateful Memes Challenge: An Enhanced Multimodal Framework

Aijing Gao, Bingjun Wang, Jiaqi Yin, Yating Tian

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Towards Robust Real-time Audio-Visual Speech Enhancement

Mandar Gogate, Kia Dashtipour, Amir Hussain

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Audio-Visual Synchronisation in the wild

Honglie Chen, Weidi Xie, Triantafyllos Afouras, Arsha Nagrani, Andrea Vedaldi, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Multi-modal Automated Speech Scoring using Attention Fusion

Manraj Singh Grover, Yaman Kumar, Sumit Sarin, Payman Vafaee, Mika Hama, Rajiv Ratn Shah

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Towards Intelligibility-Oriented Audio-Visual Speech Enhancement

Tassadaq Hussain, Mandar Gogate, Kia Dashtipour, Amir Hussain

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 6 pages, 4 figures

MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets

Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments The paper has been accepted in the Findings of Empirical Methods in Natural Language Processing (EMNLP), 2021

How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild

Okan Köpüklü, Maja Taseska, Gerhard Rigoll

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICCV 2021

The Right to Talk: An Audio-Visual Transformer Approach

Thanh-Dat Truong, Chi Nhan Duong, The De Vu, Hoang Anh Pham, Bhiksha Raj, Ngan Le, Khoa Luu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICCV 2021

Move2Hear: Active Audio-Visual Source Separation

Sagnik Majumder, Ziad Al-Halah, Kristen Grauman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICCV 2021

Continuous Emotion Recognition with Audio-visual Leader-follower Attentive Fusion

Su Zhang, Yi Ding, Ziquan Wei, Cuntai Guan

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

Comments 8 pages, 2 figures, 2 tables, accepted to the ICCV 2021: 2nd Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW2)

A Multimodal Framework for the Detection of Hateful Memes

Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, Helen Yannakoudakis

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref PMLR 133:344-360, 2021

Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers

Chiori Hori, Takaaki Hori, Jonathan Le Roux

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.CL

Comments Interspeech 2021 accepted

EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation

Hsiao-Tzu Hung, Joann Ching, Seungheon Doh, Nabin Kim, Juhan Nam, Yi-Hsuan Yang

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.MM、eess.AS

Comments The paper has been accepted for publication at ISMIR 2021

Direct multimodal few-shot learning of speech and images

Leanne Nortje, Herman Kamper

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to Interspeech 2021

Listen, Read, and Identify: Multimodal Singing Language Identification of Music

Keunwoo Choi, Yuxuan Wang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments ISMIR 2021 camera-ready

Multimodal Icon Annotation For Mobile Applications

Xiaoxue Zang, Ying Xu, Jindong Chen

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, MobileHCI 2021

Cross-Modal Knowledge Distillation Method for Automatic Cued Speech Recognition

Jianrong Wang, Ziyue Tang, Xuewei Li, Mei Yu, Qiang Fang, Li Liu

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.MM、eess.AS

Dual Normalization Multitasking for Audio-Visual Sounding Object Localization

Tokuhiro Nishikawa, Daiki Shimada, Jerry Jun Yokono

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 10 pages, 6 figures

Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds

Efthymios Tzinis, Scott Wisdom, Aren Jansen, Shawn Hershey, Tal Remez, Daniel P. W. Ellis, John R. Hershey

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments ICLR 2021, 27 pages

Naturalistic audio-visual volumetric sequences dataset of sounding actions for six degree-of-freedom interaction

Hanne Stenzel, Davide Berghi, Marco Volino, Philip J. B. Jackson

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM、eess.AS

Comments for dataset visit cvssp.org/data/navvs; accepted as poster in IEEE VR 2021

SpeakingFaces: A Large-Scale Multimodal Dataset of Voice Commands with Visual and Thermal Video Streams

Madina Abdrakhmanova, Askat Kuzdeuov, Sheikh Jarju, Yerbolat Khassanov, Michael Lewis, Huseyin Atakan Varol

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments 20 pages, 9 figures, 5 tables

Weakly-supervised Audio-visual Sound Source Detection and Separation

Tanzila Rahman, Leonid Sigal

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 4 figures, 6 pages

Journal ref IEEE International Conference on Multimedia and Expo (ICME) 2021

Enriched Music Representations with Multiple Cross-modal Contrastive Learning

Andres Ferraro, Xavier Favory, Konstantinos Drossos, Yuntae Kim, Dmitry Bogdanov

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.MM、eess.AS

Comments Accepted for publication to IEEE Signal Processing Letters

Mixture of Inference Networks for VAE-based Audio-visual Speech Enhancement

Mostafa Sadeghi, Xavier Alameda-Pineda

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments IEEE Transactions on Signal Processing

Multimodal Punctuation Prediction with Contextual Dropout

Andrew Silva, Barry-John Theobald, Nicholas Apostoloff

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication at ICASSP 2021

End-to-end Audio-visual Speech Recognition with Conformers

Pingchuan Ma, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICASSP 2021

A Fractal Approach to Characterize Emotions in Audio and Visual Domain: A Study on Cross-Modal Interaction

Sayan Nag, Uddalok Sarkar, Shankha Sanyal, Archi Banerjee, Souparno Roy, Samir Karmakar, Ranjan Sengupta, Dipak Ghosh

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL、eess.AS