多模态大模型 - arXivDaily 专题

2109.07846 2022-10-18 cs.LG cs.SD eess.AS q-bio.BM 79%

Telehealthcare and Telepathology in Pandemic: A Noninvasive, Low-Cost Micro-Invasive and Multimodal Real-Time Online Application for Early Diagnosis of COVID-19 Infection

Abdullah Bin Shams, Md. Mohsin Sarker Raihan, Md. Mohi Uddin Khan, Ocean Monjur, Rahat Bin Preo

专题命中音频语音多模态：multimodal(title,abstract)；分类 eess.AS

Comments 32 Pages. This article has been submitted for review to a prestigious journal

2111.04063 2022-10-12 eess.AS cs.SD 79%

LiMuSE: Lightweight Multi-modal Speaker Extraction

Qinghua Liu, Yating Huang, Yunzhe Hao, Jiaming Xu, Bo Xu

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

Comments Accepted to IEEE SLT 2022

2101.06268 2022-09-28 eess.AS cs.SD 79%

AMFFCN: Attentional Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement

Xinmeng Xu, Jianjun Hao

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

Comments arXiv admin note: text overlap with arXiv:2101.05975

2203.00257 2022-09-21 cs.CL 79%

Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

Yang Wu, Yanyan Zhao, Hao Yang, Song Chen, Bing Qin, Xiaohuan Cao, Wenting Zhao

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments Findings of ACL 2022

2110.09890 2022-09-19 eess.AS cs.LG cs.SD 79%

Multi-Modal Pre-Training for Automated Speech Recognition

David M. Chan, Shalini Ghosh, Debmalya Chakrabarty, Björn Hoffmeister

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

Comments Presented at ICASSP 2022

2209.03137 2022-09-08 cs.LG cs.AI 79%

Federated Transfer Learning with Multimodal Data

Yulian Sun

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Comments 73 pages, 54 figures, master thesis

2208.08909 2022-08-24 cs.HC cs.CL cs.CY 79%

"Are you okay, honey?": Recognizing Emotions among Couples Managing Diabetes in Daily Life using Multimodal Real-World Smartwatch Data

George Boateng, Xiangyu Zhao, Malgorzata Speichert, Elgar Fleisch, Janina Lüscher, Theresa Pauly, Urte Scholz, Guy Bodenmann, Tobias Kowatsch

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments 19 pages, Under review at ACM IMWUT

2208.08118 2022-08-18 cs.CV 79%

Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors

Sindhu B Hegde, Rudrabha Mukhopadhyay, Vinay P Namboodiri, C. V. Jawahar

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted in ACM-MM 2022, 10 pages, 6 pages supplementary, 18 Figures

2208.01917 2022-08-04 cs.SD cs.HC cs.LG eess.AS 79%

Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding

Mireille Fares, Michele Grimaldi, Catherine Pelachaud, Nicolas Obin

专题命中音频语音多模态：multimodal(title,abstract)；分类 eess.AS

2204.11573 2022-08-02 cs.CV 79%

Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing

Haoyue Cheng, Zhaoyang Liu, Hang Zhou, Chen Qian, Wayne Wu, Limin Wang

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2022

2104.06401 2022-07-12 cs.CV 79%

Self-supervised object detection from audio-visual correspondence

Triantafyllos Afouras, Yuki M. Asano, Francois Fagan, Andrea Vedaldi, Florian Metze

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2022

2207.01832 2022-07-06 cs.SD eess.AS 79%

Glow-WaveGAN 2: High-quality Zero-shot Text-to-speech Synthesis and Any-to-any Voice Conversion

Yi Lei, Shan Yang, Jian Cong, Lei Xie, Dan Su

专题命中音频语音多模态：any-to-any(title,abstract)；分类 eess.AS

2201.12771 2022-06-14 cs.CV 79%

Self-Supervised Moving Vehicle Detection from Audio-Visual Cues

Jannik Zürn, Wolfram Burgard

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments 8 pages, 6 figures

2206.03885 2022-06-09 eess.AS eess.SP 79%

On the Integration of Acoustics and LiDAR: a Multi-Modal Approach to Acoustic Reflector Estimation

Ellen Riemens, Pablo Martínez-Nuevo, Jorge Martinez, Martin Møller, Richard C. Hendriks

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

Comments 5 pages, 9 figures, to be published in EUSIPCO 2022

2203.14960 2022-05-24 cs.LG cs.AI 79%

Domino: Discovering Systematic Errors with Cross-Modal Embeddings

Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, Christopher Ré

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.AI

Comments ICLR 2022 (Oral)

2101.05975 2022-05-24 eess.AS cs.SD eess.IV 79%

Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement

Xinmeng Xu, Jianjun Hao

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

2205.09744 2022-05-20 cs.LG cs.CY cs.MM 79%

Overcoming Language Disparity in Online Content Classification with Multimodal Learning

Gaurav Verma, Rohit Mujumdar, Zijie J. Wang, Munmun De Choudhury, Srijan Kumar

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.MM

Comments Accepted for publication at ICWSM 2022 as a full paper

2008.09743 2022-05-16 cs.SD cs.CV 79%

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Guanghao Yin, Shouqian Sun, Dian Yu, Dejian Li, Kejun Zhang

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

Comments ACM Transactions on Multimedia Computing, Communications, and Applications (Acceptance 07-Oct-2021)

2205.00073 2022-05-03 cs.CV 79%

On Negative Sampling for Audio-Visual Contrastive Learning from Movies

Mahdi M. Kalayeh, Shervin Ardeshir, Lingyi Liu, Nagendra Kamath, Ashok Chandrashekar

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2106.08513

2204.12366 2022-04-27 cs.MM 79%

Robust Audio-Visual Instance Discrimination via Active Contrastive Set Mining

Hanyu Xuan, Yihong Xu, Shuo Chen, Zhiliang Wu, Jian Yang, Yan Yan, Xavier Alameda-Pineda

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.MM

Comments 7 pages, 4 figures, accepted at IJCAI 2022

2204.08686 2022-04-21 cs.SD eess.AS 79%

Audio-Visual Wake Word Spotting System For MISP Challenge 2021

Yanguang Xu, Jianwei Sun, Yang Han, Shuaijiang Zhao, Chaoyang Mei, Tingwei Guo, Shuran Zhou, Chuandong Xie, Wei Zou, Xiangang Li, Shuran Zhou, Chuandong Xie, Wei Zou, Xiangang Li

专题命中音频语音多模态：audio-visual(title)；multimodal(abstract)；分类 eess.AS

Comments Accepted to ICASSP 2022

2110.10101 2022-04-21 cs.CV 79%

Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition

Mirco Planamente, Chiara Plizzari, Emanuele Alberti, Barbara Caputo

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted at WACV 2022. arXiv admin note: substantial text overlap with arXiv:2106.01689

2202.04303 2022-04-20 cs.LG cs.SD eess.AS 79%

TinyM$^2$Net: A Flexible System Algorithm Co-designed Multimodal Learning Framework for Tiny Devices

Hasib-Al Rashid, Pretom Roy Ovi, Carl Busart, Aryya Gangopadhyay, Tinoosh Mohsenin

专题命中音频语音多模态：multimodal(title,abstract)；分类 eess.AS

Comments tinyML Research Symposium 2022

2204.02905 2022-04-07 cs.CL cs.HC 79%

EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios

Sunit Bhattacharya, Věra Kloudová, Vilém Zouhar, Ondřej Bojar

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CL

Comments Submitted to Nature Scientific Data

2203.13032 2022-04-01 cs.CV eess.IV 79%

Multi-modal Emotion Estimation for in-the-wild Videos

Liyu Meng, Yuchen Liu, Xiaolong Liu, Zhaopei Huang, Yuan Cheng, Meng Wang, Chuanhe Liu, Qin Jin

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CV

2203.00674 2022-03-02 cs.CL cs.CY 79%

Advancing an Interdisciplinary Science of Conversation: Insights from a Large Multimodal Corpus of Human Speech

Andrew Reece, Gus Cooney, Peter Bull, Christine Chung, Bryn Dawson, Casey Fitzpatrick, Tamara Glazer, Dean Knox, Alex Liebscher, Sebastian Marin

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments 116 pages; Cross-posted to PsyArxiv; To request data access, go to https://betterup-data-requests.herokuapp.com/

2202.10976 2022-02-23 cs.SD eess.AS 79%

DRVC: A Framework of Any-to-Any Voice Conversion with Self-Supervised Learning

Qiqi Wang, Xulong Zhang, Jianzong Wang, Ning Cheng, Jing Xiao

专题命中音频语音多模态：any-to-any(title,abstract)；分类 eess.AS

Comments Published at ICASSP 2022

2112.15110 2022-02-23 cs.SD cs.LG eess.AS 79%

Audio-to-symbolic Arrangement via Cross-modal Music Representation Learning

Ziyu Wang, Dejing Xu, Gus Xia, Ying Shan

专题命中音频语音多模态：cross-modal(title,abstract)；分类 eess.AS

2202.09195 2022-02-21 cs.LG cs.MM 79%

A Review on Methods and Applications in Multimodal Deep Learning

Jabeen Summaira, Xi Li, Amin Muhammad Shoib, Jabbar Abdul

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.MM

Comments 29 pages. arXiv admin note: substantial text overlap with arXiv:2105.11087

Journal ref ACM Transactions on Multimedia Computing, Communications, and Applications 2022

2202.08974 2022-02-21 cs.SD cs.HC cs.LG cs.RO eess.AS 79%

Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models

Sarala Padi, Seyed Omid Sadjadi, Dinesh Manocha, Ram D. Sriram

专题命中音频语音多模态：multimodal(title,abstract)；分类 eess.AS

Comments arXiv admin note: substantial text overlap with arXiv:2108.02510

1. 音频语音多模态 4559 篇

Telehealthcare and Telepathology in Pandemic: A Noninvasive, Low-Cost Micro-Invasive and Multimodal Real-Time Online Application for Early Diagnosis of COVID-19 Infection

LiMuSE: Lightweight Multi-modal Speaker Extraction

AMFFCN: Attentional Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement

Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

Multi-Modal Pre-Training for Automated Speech Recognition

Federated Transfer Learning with Multimodal Data

"Are you okay, honey?": Recognizing Emotions among Couples Managing Diabetes in Daily Life using Multimodal Real-World Smartwatch Data

Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors

Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding

Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing

Self-supervised object detection from audio-visual correspondence

Glow-WaveGAN 2: High-quality Zero-shot Text-to-speech Synthesis and Any-to-any Voice Conversion

Self-Supervised Moving Vehicle Detection from Audio-Visual Cues

On the Integration of Acoustics and LiDAR: a Multi-Modal Approach to Acoustic Reflector Estimation

Domino: Discovering Systematic Errors with Cross-Modal Embeddings

Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement

Overcoming Language Disparity in Online Content Classification with Multimodal Learning

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

On Negative Sampling for Audio-Visual Contrastive Learning from Movies

Robust Audio-Visual Instance Discrimination via Active Contrastive Set Mining

Audio-Visual Wake Word Spotting System For MISP Challenge 2021

Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition

TinyM$^2$Net: A Flexible System Algorithm Co-designed Multimodal Learning Framework for Tiny Devices

EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios

Multi-modal Emotion Estimation for in-the-wild Videos

Advancing an Interdisciplinary Science of Conversation: Insights from a Large Multimodal Corpus of Human Speech

DRVC: A Framework of Any-to-Any Voice Conversion with Self-Supervised Learning

Audio-to-symbolic Arrangement via Cross-modal Music Representation Learning

A Review on Methods and Applications in Multimodal Deep Learning

Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models