多模态大模型

ALO-VC: Any-to-any Low-latency One-shot Voice Conversion

Bohan Wang, Damien Ronssin, Milos Cernak

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments Accepted to Interspeech 2023. Some audio samples are available at https://bohan7.github.io/ALO-VC-demo/

Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model

Héctor Martel, Julius Richter, Kai Li, Xiaolin Hu, Timo Gerkmann

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted by Interspeech 2023

Masked Audio Text Encoders are Effective Multi-Modal Rescorers

Jinglun Cai, Monica Sunkara, Xilai Li, Anshu Bhatia, Xiao Pan, Sravan Bodapati

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL

Audio-Visual Dataset and Method for Anomaly Detection in Traffic Videos

Błażej Leporowski, Arian Bakhtiarnia, Nicole Bonnici, Adrian Muscat, Luca Zanella, Yiming Wang, Alexandros Iosifidis

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis

Jieyeon Woo, Mireille Fares, Catherine Pelachaud, Catherine Achard

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments 8 pages, 1 figure

Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data

Yangshuo He, Guanding Yu, Yunlong Cai

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.AI

AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction

Aggelina Chatziagapi, Dimitris Samaras

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023. Project page: https://aggelinacha.github.io/AVFace/

Brain Connectivity Features-based Age Group Classification using Temporal Asynchrony Audio-Visual Integration Task

Prerna Singh, Ayush Tripathi, Lalan Kumar, Tapan Kumar Gandhi

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

LIPSFUS: A neuromorphic dataset for audio-visual sensory fusion of lip reading

Antonio Rios-Navarro, Enrique Piñero-Fuentes, Salvador Canas-Moreno, Aqib Javed, Jin Harkin, Alejandro Linares-Barranco

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Submitted to ISCAS2023, 4 pages, plus references, github link provided

Dance Style Transfer with Cross-modal Transformer

Wenjie Yin, Hang Yin, Kim Baraka, Danica Kragic, Mårten Björkman

专题命中 音频语音多模态 ：cross-modal(title)；multimodal(abstract)；分类 cs.MM

TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models

Md Kamrul Hasan, Md Saiful Islam, Sangwu Lee, Wasifur Rahman, Iftekhar Naim, Mohammed Ibrahim Khan, Ehsan Hoque

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Self-Supervised Video Forensics by Audio-Visual Anomaly Detection

Chao Feng, Ziyang Chen, Andrew Owens

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments CVPR 2023

Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning

Weixuan Sun, Jiayi Zhang, Jianyuan Wang, Zheyuan Liu, Yiran Zhong, Tianpeng Feng, Yandong Guo, Yanhao Zhang, Nick Barnes

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments CVPR2023

Dialogue Enhancement and Listening Effort in Broadcast Audio: A Multimodal Evaluation

Matteo Torcoli, Thomas Robotham, Emanuël A. P. Habets

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Paper accepted to 14th International Conference on Quality of Multimedia Experience (QoMEX), Lippstadt, Germany, 2022 - version 2 fixes some typos

Multi-modal Expression Recognition with Ensemble Method

Chuanhe Liu, Xinjie Zhang, Xiaolong Liu, Tenggan Zhang, Liyu Meng, Yuchen Liu, Yuanyuan Deng, Wenqiang Jiang

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

TriAAN-VC: Triple Adaptive Attention Normalization for Any-to-Any Voice Conversion

Hyun Joon Park, Seok Woo Yang, Jin Sob Kim, Wooseok Shin, Sung Won Han

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments To appear in ICASSP 2023

A large-scale multimodal dataset of human speech recognition

Yao Ge, Chong Tang, Haobo Li, Zikang Zhang, Wenda Li, Kevin Chetty, Daniele Faccio, Qammer H. Abbasi, Muhammad Imran

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Real-Time Audio-Visual End-to-End Speech Enhancement

Zirun Zhu, Hemin Yang, Min Tang, Ziyi Yang, Sefik Emre Eskimez, Huaming Wang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted by ICASSP 2023

The DKU Post-Challenge Audio-Visual Wake Word Spotting System for the 2021 MISP Challenge: Deep Analysis

Haoxu Wang, Ming Cheng, Qiang Fu, Ming Li

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 eess.AS

Comments Accepted by ICASSP 2023

Key-Sparse Transformer for Multimodal Speech Emotion Recognition

Weidong Chen, Xiaofeng Xing, Xiangmin Xu, Jichen Yang, Jianxin Pang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments This paper was accepted by ICASSP 2022

MM-ALT: A Multimodal Automatic Lyric Transcription System

Xiangming Gu, Longshen Ou, Danielle Ong, Ye Wang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Accepted by ACM Multimedia 2022. Camera ready version and appendix

Audio-Visual Segmentation with Semantics

Jinxing Zhou, Xuyang Shen, Jianyuan Wang, Jiayi Zhang, Weixuan Sun, Jing Zhang, Stan Birchfield, Dan Guo, Lingpeng Kong, Meng Wang, Yiran Zhong

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Submitted to TPAMI as a journal extension of ECCV 2022. Jinxing Zhou, Xuyang Shen, and Jianyuan Wang contribute equally to this work. Meng Wang and Yiran Zhong are the corresponding authors. Code is available at https://github.com/OpenNLPLab/AVSBench. Online benchmark is available at http://www.avlbench.opennlplab.cn. arXiv admin note: substantial text overlap with arXiv:2207.05042

Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classification

Liguang Zhou, Yuhongze Zhou, Xiaonan Qi, Junjie Hu, Tin Lun Lam, Yangsheng Xu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work

Pritam Sarkar, Aaron Posen, Ali Etemad

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted in AAAI 2023

Self-supervised Learning of Audio Representations from Audio-Visual Data using Spatial Alignment

Shanshan Wang, Archontis Politis, Annamaria Mesaros, Tuomas Virtanen

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Contrastive Regularization for Multimodal Emotion Recognition Using Audio and Text

Fan Qian, Jiqing Han

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Completed in October 2020 and submitted to ICASSP2021

Contrastive Positive Sample Propagation along the Audio-Visual Event Line

Jinxing Zhou, Dan Guo, Meng Wang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted to TPAMI; Dataset and Code are available at https://github.com/jasongief/CPSP. arXiv admin note: substantial text overlap with arXiv:2104.00239

Vis2Mus: Exploring Multimodal Representation Mapping for Controllable Music Generation

Runbang Zhang, Yixiao Zhang, Kai Shao, Ying Shan, Gus Xia

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Submitted to ICASSP 2023. GitHub repo: https://github.com/ldzhangyx/vis2mus

Self-Supervised Training of Speaker Encoder with Multi-Modal Diverse Positive Pairs

Ruijie Tao, Kong Aik Lee, Rohan Kumar Das, Ville Hautamäki, Haizhou Li

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments 13 pages

Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories

Fábio Vital, Miguel Vasco, Alberto Sardinha, Francisco Melo

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 14 pages, 4 figures, 8 tables, 1 algorithm