多模态大模型 - arXivDaily 专题

2203.14072 2022-04-06 cs.CV 83%

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, Di Hu

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted by CVPR2022 (Oral presentation)

2202.09263 2022-02-21 cs.LG cs.MM 83%

Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?

Vandana Rajan, Alessio Brutti, Andrea Cavallaro

专题命中音频语音多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted at ICASSP 2022

2202.09081 2022-02-21 eess.AS cs.AI cs.CV cs.MM cs.SD eess.IV 83%

VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion

Disong Wang, Shan Yang, Dan Su, Xunying Liu, Dong Yu, Helen Meng

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to ICASSP 2022. Demo page is available at https://wendison.github.io/VCVTS-demo/

2202.05662 2022-02-14 cs.CR cs.SD eess.AS 83%

A Novel Chaos-based Light-weight Image Encryption Scheme for Multi-modal Hearing Aids

Awais Aziz Shah, Ahsan Adeel, Jawad Ahmad, Ahmed Al-Dubai, Mandar Gogate, Abhijeet Bishnu, Muhammad Diyan, Tassadaq Hussain, Kia Dashtipour, Tharm Ratnarajah, Amir Hussain

专题命中音频语音多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 eess.AS

2008.13222 2022-02-02 eess.AS cs.LG eess.IV 83%

Improved Lite Audio-Visual Speech Enhancement

Shang-Yi Chuang, Hsin-Min Wang, Yu Tsao

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 eess.AS

2112.14298 2021-12-30 cs.CV cs.RO 83%

Multimodal perception for dexterous manipulation

Guanqun Cao, Shan Luo

专题命中音频语音多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 19 pages, 10 figures

1907.01826 2021-12-13 cs.CV 83%

Cascade Attention Guided Residue Learning GAN for Cross-Modal Translation

Bin Duan, Wei Wang, Hao Tang, Hugo Latapie, Yan Yan

专题命中音频语音多模态：cross-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV

Comments 9 pages, 6 figures, update template

2109.04138 2021-11-16 cs.CR cs.CV 83%

Multilingual Audio-Visual Smartphone Dataset And Evaluation

Hareesh Mandalapu, Aravinda Reddy P N, Raghavendra Ramachandra, K Sreenivasa Rao, Pabitra Mitra, S R Mahadeva Prasanna, Christoph Busch

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

2111.04823 2021-11-10 cs.CL cs.CV cs.MM cs.SD eess.AS eess.IV 83%

Cascaded Multilingual Audio-Visual Learning from Videos

Andrew Rouditchenko, Angie Boggust, David Harwath, Samuel Thomas, Hilde Kuehne, Brian Chen, Rameswar Panda, Rogerio Feris, Brian Kingsbury, Michael Picheny, James Glass

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Presented at Interspeech 2021. This version contains updated results using the YouCook-Japanese dataset

2108.07897 2021-11-10 cs.CV cs.LG 83%

Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection

Leena Mathur, Maja J Matarić

专题命中音频语音多模态：multimodal(title,abstract)；audio-visual(abstract)；分类 cs.CV

2110.14862 2021-10-29 cs.CV 83%

Audio-visual Representation Learning for Anomaly Events Detection in Crowds

Junyu Gao, Maoguo Gong, Xuelong Li

专题命中音频语音多模态：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 6 figures

2104.09946 2021-10-20 cs.SD cs.LG eess.AS 83%

A cappella: Audio-visual Singing Voice Separation

Juan F. Montesinos, Venkatesh S. Kadandale, Gloria Haro

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 eess.AS

Comments Paper accepted at The 32nd British Machine Vision Conference, BMVC 2021

2106.09669 2021-10-15 cs.SD cs.CV cs.LG 83%

Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention

Efthymios Tzinis, Scott Wisdom, Tal Remez, John R. Hershey

专题命中音频语音多模态：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.CV

2110.06894 2021-10-14 cs.CL 83%

Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning

Ankit P. Shah, Shijie Geng, Peng Gao, Anoop Cherian, Takaaki Hori, Tim K. Marks, Jonathan Le Roux, Chiori Hori

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CL

Comments https://dstc10.dstc.community/home and https://github.com/dialogtekgeek/AVSD-DSTC10_Official/

2110.05122 2021-10-12 cs.CV 83%

Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos

Heeseung Yun, Youngjae Yu, Wonsuk Yang, Kangil Lee, Gunhee Kim

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Published to ICCV2021

2101.03682 2021-10-06 cs.CV 83%

MAAS: Multi-modal Assignation for Active Speaker Detection

Juan León-Alcázar, Fabian Caba Heilbron, Ali Thabet, Bernard Ghanem

专题命中音频语音多模态：multi-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV

2109.04894 2021-09-13 eess.AS cs.SD eess.IV 83%

Large-vocabulary Audio-visual Speech Recognition in Noisy Environments

Wentao Yu, Steffen Zeiler, Dorothea Kolossa

专题命中音频语音多模态：audio-visual(title,abstract)；multi-modal(abstract)；分类 eess.AS

Journal ref The IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), 2021

2108.11773 2021-08-31 cs.CV 83%

Multi-Modulation Network for Audio-Visual Event Localization

Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, Jiebo Luo

专题命中音频语音多模态：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.CV

2105.13675 2021-07-21 eess.AS cs.SD 83%

Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

Shanshan Wang, Toni Heittola, Annamaria Mesaros, Tuomas Virtanen

专题命中音频语音多模态：audio-visual(title,abstract)；multi-modal(abstract)；分类 eess.AS

2102.03673 2021-06-22 cs.CV cs.HC cs.LG 83%

Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection

Leena Mathur, Maja J Matarić

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted at ICASSP 2021 \c{opyright} 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of copyrighted components of this work

2007.10695 2021-06-22 cs.LG cs.MM stat.ML 83%

Towards Multimodal MIR: Predicting individual differences from music-induced movement

Yudhik Agrawal, Samyak Jain, Emily Carlson, Petri Toiviainen, Vinoo Alluri

专题命中音频语音多模态：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.MM

Comments Appearing in the proceedings of the 21st International Society for Music Information Retrieval Conference (ISMIR 2020) (camera-ready version)

Journal ref International Society for Music Information Retrieval Conference, (2020) 54-61

2106.01689 2021-06-04 cs.CV 83%

Cross-Domain First Person Audio-Visual Action Recognition through Relative Norm Alignment

Mirco Planamente, Chiara Plizzari, Emanuele Alberti, Barbara Caputo

专题命中音频语音多模态：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 11 pages, 7 figures

2007.12519 2021-04-23 cs.CV cs.LG 83%

HEU Emotion: A Large-scale Database for Multi-modal Emotion Recognition in the Wild

Jing Chen, Chenhui Wang, Kejun Wang, Chaoqun Yin, Cong Zhao, Tao Xu, Xinyi Zhang, Ziqiang Huang, Meichen Liu, Tao Yang

专题命中音频语音多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Neural Comput & Applic (2021)

2103.15916 2021-03-31 cs.CV 83%

Robust Audio-Visual Instance Discrimination

Pedro Morgado, Ishan Misra, Nuno Vasconcelos

专题命中音频语音多模态：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.CV

2008.09586 2021-03-16 eess.AS cs.LG eess.IV 83%

An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation

Daniel Michelsanti, Zheng-Hua Tan, Shi-Xiong Zhang, Yong Xu, Meng Yu, Dong Yu, Jesper Jensen

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 eess.AS

2102.06291 2021-02-15 cs.SD cs.LG eess.AS eess.IV 83%

A Multi-View Approach To Audio-Visual Speaker Verification

Leda Sarı, Kritika Singh, Jiatong Zhou, Lorenzo Torresani, Nayan Singhal, Yatharth Saraf

专题命中音频语音多模态：audio-visual(title,abstract)；cross-modal(abstract)；分类 eess.AS

2102.06269 2021-02-15 eess.IV cs.SD eess.AS 83%

Disentanglement for audio-visual emotion recognition using multitask setup

Raghuveer Peri, Srinivas Parthasarathy, Charles Bradshaw, Shiva Sundaram

专题命中音频语音多模态：audio-visual(title,abstract)；multimodal(abstract)；分类 eess.AS

Comments Accepted for ICASSP 2021, 5 pages

2102.01326 2021-02-03 eess.AS cs.LG cs.SD 83%

Multimodal Attention Fusion for Target Speaker Extraction

Hiroshi Sato, Tsubasa Ochiai, Keisuke Kinoshita, Marc Delcroix, Tomohiro Nakatani, Shoko Araki

专题命中音频语音多模态：multimodal(title)；multi-modal(abstract)；audio-visual(abstract)；分类 eess.AS

Comments 7 pages, 5 figures

Journal ref in IEEE Spoken Language Technology Workshop (SLT), 2021, pp. 778-784

2012.00063 2020-12-02 eess.AS cs.SD eess.IV 83%

Detecting expressions with multimodal transformers

Srinivas Parthasarathy, Shiva Sundaram

专题命中音频语音多模态：multimodal(title,abstract)；audio-visual(abstract)；分类 eess.AS

Comments IEEE Spoken Language Technology Workshop 2021

2006.16228 2020-11-02 cs.CV 83%

Self-Supervised MultiModal Versatile Networks

Jean-Baptiste Alayrac, Adrià Recasens, Rosalia Schneider, Relja Arandjelović, Jason Ramapuram, Jeffrey De Fauw, Lucas Smaira, Sander Dieleman, Andrew Zisserman

专题命中音频语音多模态：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments To appear in the Thirty-Fourth Annual Conference on Neural Information Processing Systems (NeurIPS 2020)

1. 音频语音多模态 4549 篇

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?

VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion

A Novel Chaos-based Light-weight Image Encryption Scheme for Multi-modal Hearing Aids

Improved Lite Audio-Visual Speech Enhancement

Multimodal perception for dexterous manipulation

Cascade Attention Guided Residue Learning GAN for Cross-Modal Translation

Multilingual Audio-Visual Smartphone Dataset And Evaluation

Cascaded Multilingual Audio-Visual Learning from Videos

Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection

Audio-visual Representation Learning for Anomaly Events Detection in Crowds

A cappella: Audio-visual Singing Voice Separation

Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention

Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning

Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos

MAAS: Multi-modal Assignation for Active Speaker Detection

Large-vocabulary Audio-visual Speech Recognition in Noisy Environments

Multi-Modulation Network for Audio-Visual Event Localization

Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection

Towards Multimodal MIR: Predicting individual differences from music-induced movement

Cross-Domain First Person Audio-Visual Action Recognition through Relative Norm Alignment

HEU Emotion: A Large-scale Database for Multi-modal Emotion Recognition in the Wild

Robust Audio-Visual Instance Discrimination

An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation

A Multi-View Approach To Audio-Visual Speaker Verification

Disentanglement for audio-visual emotion recognition using multitask setup

Multimodal Attention Fusion for Target Speaker Extraction

Detecting expressions with multimodal transformers

Self-Supervised MultiModal Versatile Networks