多模态大模型

Late Audio-Visual Fusion for In-The-Wild Speaker Diarization

Zexu Pan, Gordon Wichern, François G. Germain, Aswin Subramanian, Jonathan Le Roux

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS

Multimodal Modeling For Spoken Language Identification

Shikhar Bharadwaj, Min Ma, Shikhar Vashishth, Ankur Bapna, Sriram Ganapathy, Vera Axelrod, Siddharth Dalmia, Wei Han, Yu Zhang, Daan van Esch, Sandy Ritchie, Partha Talukdar, Jason Riesa

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

PIAVE: A Pose-Invariant Audio-Visual Speaker Extraction Network

Qinghua Liu, Meng Ge, Zhizheng Wu, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM、eess.AS

Comments Interspeech 2023

Journal ref Proc. INTERSPEECH 2023, 3719-3723

DF-TransFusion: Multimodal Deepfake Detection via Lip-Audio Cross-Attention and Facial Self-Attention

Aaditya Kharel, Manas Paranjape, Aniket Bera

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.MM

Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues

Akira Taniguchi, Hiroaki Murakami, Ryo Ozaki, Tadahiro Taniguchi

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to IEEE TRANSACTIONS ON COGNITIVE DEVELOPMENTAL SYSTEMS

Multi-goal Audio-visual Navigation using Sound Direction Map

Haru Kondoh, Asako Kanezaki

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments IROS2023

Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers

Syed Aun Muhammad Zaidi, Siddique Latif, Junaid Qadir

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Under Review IEEE TAC

Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition

Guinan Li, Jiajun Deng, Mengzhe Geng, Zengrui Jin, Tianzi Wang, Shujie Hu, Mingyu Cui, Helen Meng, Xunying Liu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI、eess.AS

Comments IEEE/ACM Transactions on Audio, Speech, and Language Processing

Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels

Pingchuan Ma, Alexandros Haliassos, Adriana Fernandez-Lopez, Honglie Chen, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICASSP 2023

AV-SepFormer: Cross-Attention SepFormer for Audio-Visual Target Speaker Extraction

Jiuxin Lin, Xinyu Cai, Heinrich Dinkel, Jun Chen, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Zhiyong Wu, Yujun Wang, Helen Meng

专题命中 音频语音多模态 ：audio-visual(title)；multi-modal(abstract)；分类 cs.MM、eess.AS

Comments Accepted by ICASSP2023

Multimodal Audio-textual Architecture for Robust Spoken Language Understanding

Anderson R. Avila, Mehdi Rezagholizadeh, Chao Xing

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus

Théo Deschamps-Berger, Lori Lamel, Laurence Devillers

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments 5 pages, 2 figures, 4 tables

Journal ref Published in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment

Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, Yongbin Li

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACL 2023 main conference

BeAts: Bengali Speech Acts Recognition using Multimodal Attention Fusion

Ahana Deb, Sayan Nag, Ayan Mahapatra, Soumitri Chattopadhyay, Aritra Marik, Pijush Kanti Gayen, Shankha Sanyal, Archi Banerjee, Samir Karmakar

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted at INTERSPEECH 2023

Encoder-decoder multimodal speaker change detection

Jee-weon Jung, Soonshin Seo, Hee-Soo Heo, Geonmin Kim, You Jin Kim, Young-ki Kwon, Minjae Lee, Bong-Jin Lee

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

Comments 5 pages, accepted for presentation at INTERSPEECH 2023

MPE4G: Multimodal Pretrained Encoder for Co-Speech Gesture Generation

Gwantae Kim, Seonghyeok Noh, Insung Ham, Hanseok Ko

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 3 figures

Journal ref ICASSP 2023

Rethinking Speech Recognition with A Multimodal Perspective via Acoustic and Semantic Cooperative Decoding

Tian-Hao Zhang, Hai-Bo Qin, Zhi-Hao Lai, Song-Lu Chen, Qi Liu, Feng Chen, Xinyuan Qian, Xu-Cheng Yin

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted by Interspeech 2023

ZS-MSTM: Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding

Mireille Fares, Catherine Pelachaud, Nicolas Obin

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

Comments arXiv admin note: substantial text overlap with arXiv:2208.01917

Audio-Visual Person-of-Interest DeepFake Detection

Davide Cozzolino, Alessandro Pianese, Matthias Nießner, Luisa Verdoliva

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models

Sixia Li, Shogo Okada

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments 4 tables, 4 figures

M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis

Jinlong Xue, Yayue Deng, Fengping Wang, Ya Li, Yingming Gao, Jianhua Tao, Jianqing Sun, Jiaen Liang

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 5 pages, 1 figures, 2 tables. Accepted by ICASSP 2023

Learning Multimodal Data Augmentation in Feature Space

Zichang Liu, Zhiqiang Tang, Xingjian Shi, Aston Zhang, Mu Li, Anshumali Shrivastava, Andrew Gordon Wilson

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ICLR 2023. Code available at https://github.com/lzcemma/LeMDA/

Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning

Hao Zhang, Nianwen Si, Yaqi Chen, Wenlin Zhang, Xukui Yang, Dan Qu, Wei-Qiang Zhang

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Journal ref IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 31, 2023

Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5

Su Zhang, Ziyuan Zhao, Cuntai Guan

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM、eess.AS

Comments 6 pages. 1 figure. arXiv admin note: substantial text overlap with arXiv:2203.13031

Multimodal and Explainable Internet Meme Classification

Abhinav Kumar Thakur, Filip Ilievski, Hông-Ân Sandlin, Zhivar Sourati, Luca Luceri, Riccardo Tommasini, Alain Mermoud

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Audio-Visual Grouping Network for Sound Localization from Mixtures

Shentong Mo, Yapeng Tian

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

Comments CVPR 2023

Practice of the conformer enhanced AUDIO-VISUAL HUBERT on Mandarin and English

Xiaoming Ren, Chao Li, Shenjian Wang, Biao Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI、eess.AS

Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations

Se-Yun Um, Jihyun Kim, Jihyun Lee, Hong-Goo Kang

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV、eess.AS

Comments 5 pages (including references), 1 figure

LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders

Rodrigo Mira, Buye Xu, Jacob Donley, Anurag Kumar, Stavros Petridis, Vamsi Krishna Ithapu, Maja Pantic

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments accepted to ICASSP 2023

MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation

Mohamed Anwar, Bowen Shi, Vedanuj Goswami, Wei-Ning Hsu, Juan Pino, Changhan Wang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS