多模态大模型

Learning to Set Waypoints for Audio-Visual Navigation

Changan Chen, Sagnik Majumder, Ziad Al-Halah, Ruohan Gao, Santhosh Kumar Ramakrishnan, Kristen Grauman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2021

Muse: Multi-modal target speaker extraction with visual cues

Zexu Pan, Ruijie Tao, Chenglin Xu, Haizhou Li

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.MM、eess.AS

Comments Accepted by ICASSP2021

Switching Variational Auto-Encoders for Noise-Agnostic Audio-visual Speech Enhancement

Mostafa Sadeghi, Xavier Alameda-Pineda

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Face Landmark-based Speaker-Independent Audio-Visual Speech Enhancement in Multi-Talker Environments

Giovanni Morrone, Luca Pasa, Vadim Tikhanoff, Sonia Bergamaschi, Luciano Fadiga, Leonardo Badino

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS

Comments Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

See, Hear, Explore: Curiosity via Audio-Visual Association

Victoria Dean, Shubham Tulsiani, Abhinav Gupta

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.AI

QUARC: Quaternion Multi-Modal Fusion Architecture For Hate Speech Classification

Deepak Kumar, Nalin Kumar, Subhankar Mishra

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted in Proc. of the 4th International Workshop on Dialog Systems (IWDS2021) in conjunction with the IEEE BigComp2021

Audio-visual Speech Separation with Adversarially Disentangled Visual Representation

Peng Zhang, Jiaming Xu, Jing shi, Yunzhe Hao, Bo Xu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 5 pages, 3 figures

Audio-visual Multi-channel Recognition of Overlapped Speech

Jianwei Yu, Bo Wu, Rongzhi Gu, Shi-Xiong Zhang, Lianwu Chen, Yong Xu. Meng Yu, Dan Su, Dong Yu, Xunying Liu, Helen Meng

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS

Comments submitted to Interspeech 2020

Audio-Visual Event Recognition through the lens of Adversary

Juncheng B Li, Kaixin Ma, Shuhui Qu, Po-Yao Huang, Florian Metze

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 cs.CV、eess.AS

Comments 4 pages

Multi-Modal Emotion Detection with Transfer Learning

Amith Ananthram, Kailash Karthik Saravanakumar, Jessica Huynh, Homayoon Beigi

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 11 pages, 7 tables, 2 figures

Cosine Similarity of Multimodal Content Vectors for TV Programmes

Saba Nazir, Taner Cagali, Chris Newell, Mehrnoosh Sadrzadeh

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments 3 pages, 1 figure, Machine Learning for Media Discovery (ML4MD) Workshop at ICML 2020

Augmenting Images for ASR and TTS through Single-loop and Dual-loop Multimodal Chain Framework

Johanes Effendi, Andros Tjandra, Sakriani Sakti, Satoshi Nakamura

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted at INTERSPEECH 2020

AVECL-UMONS database for audio-visual event classification and localization

Mathilde Brousmiche, Stéphane Dupont, Jean Rouat

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

VGGSound: A Large-scale Audio-Visual Dataset

Honglie Chen, Weidi Xie, Andrea Vedaldi, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments ICASSP2020

Crossing You in Style: Cross-modal Style Transfer from Music to Visual Arts

Cheng-Che Lee, Wan-Yi Lin, Yen-Ting Shih, Pei-Yi Patricia Kuo, Li Su

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Multi-modal Attention for Speech Emotion Recognition

Zexu Pan, Zhaojie Luo, Jichen Yang, Haizhou Li

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.MM、eess.AS

Comments Accepted by Interspeech2020

Lite Audio-Visual Speech Enhancement

Shang-Yi Chuang, Yu Tsao, Chen-Chou Lo, Hsin-Min Wang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to Interspeech 2020

Self-Supervised Learning of Audio-Visual Objects from Video

Triantafyllos Afouras, Andrew Owens, Joon Son Chung, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments ECCV 2020

Speech to Text Adaptation: Towards an Efficient Cross-Modal Distillation

Won Ik Cho, Donghyun Kwak, Ji Won Yoon, Nam Soo Kim

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL、eess.AS

Comments Interspeech 2020 Camera-ready

Attentive Fusion Enhanced Audio-Visual Encoding for Transformer Based Robust Speech Recognition

Liangfa Wei, Jie Zhang, Junfeng Hou, Lirong Dai

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM、eess.AS

Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speech

Monica Sunkara, Srikanth Ronanki, Dhanush Bekal, Sravan Bodapati, Katrin Kirchhoff

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted for Interspeech 2020

Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge

Hoang Manh Hung, Hyung-Jeong Yang, Soo-Hyung Kim, Guee-Sang Lee

专题命中 音频语音多模态 ：multimodal(title)；audio-visual(abstract)；分类 cs.CV、cs.CL

Comments 3 pages, 2 figures

MultiQT: Multimodal Learning for Real-Time Question Tracking in Speech

Jakob D. Havtorn, Jan Latko, Joakim Edin, Lasse Borgholt, Lars Maaløe, Lorenzo Belgrano, Nicolai F. Jacobsen, Regitze Sdun, Željko Agić

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted at ACL 2020

ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents

Chia-Yu Li, Daniel Ortega, Dirk Väth, Florian Lux, Lindsey Vanderlyn, Maximilian Schmidt, Michael Neumann, Moritz Völkel, Pavel Denisov, Sabrina Jenne, Zorica Kacarevic, Ngoc Thang Vu

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments All authors contributed equally. Accepted to be presented at ACL - System demonstrations - 2020

Learning Alignment for Multimodal Emotion Recognition from Speech

Haiyang Xu, Hui Zhang, Kun Han, Yun Wang, Yiping Peng, Xiangang Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments InterSpeech 2019

ASR is all you need: cross-modal distillation for lip reading

Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV、eess.AS

Comments ICASSP 2020

How deep is your encoder: an analysis of features descriptors for an autoencoder-based audio-visual quality metric

Helard Martinez, Andrew Hines, Mylene C. Q. Farias

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge

Hung Le, Nancy F. Chen

专题命中 音频语音多模态 ：multimodal(title)；audio-visual(abstract)；分类 cs.CV、cs.CL

Comments Accepted at DSTC Workshop at AAAI 2020

$M^3$T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild

Yuan-Hang Zhang, Rulin Huang, Jiabei Zeng, Shiguang Shan, Xilin Chen

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV、eess.AS

Comments 6 pages, technical report; submission to ABAW Challenge at FG 2020

Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions

Tejas Srinivasan, Ramon Sanabria, Florian Metze

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to How2 Workshop, ICML 2019