多模态大模型 - arXivDaily 专题

1904.08248 2019-11-28 eess.AS cs.CL cs.SD stat.ML 81%

An Analysis of Speech Enhancement and Recognition Losses in Limited Resources Multi-talker Single Channel Audio-Visual ASR

Luca Pasa, Giovanni Morrone, Leonardo Badino

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CL、eess.AS

1901.02839 2019-11-20 cs.HC cs.AI cs.CV 81%

SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild

Jean Kossaifi, Robert Walecki, Yannis Panagakis, Jie Shen, Maximilian Schmitt, Fabien Ringeval, Jing Han, Vedhas Pandit, Antoine Toisoul, Bjorn Schuller, Kam Star, Elnar Hajiyev, Maja Pantic

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、cs.AI

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019

1809.10961 2019-10-30 cs.CV cs.MM stat.ML 81%

Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers

Yutong Ban, Xavier Alameda-Pineda, Laurent Girin, Radu Horaud

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、cs.MM

1910.03814 2019-10-10 cs.CV cs.CL 81%

Exploring Hate Speech Detection in Multimodal Publications

Raul Gomez, Jaume Gibert, Lluis Gomez, Dimosthenis Karatzas

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL

1910.00330 2019-10-02 cs.LG cs.CL eess.AS stat.ML 81%

A Multi-Modal Feature Embedding Approach to Diagnose Alzheimer Disease from Spoken Language

S. Soroush Haj Zargarbashi, Bagher Babaali

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 14 pages, 4 figures

1909.10407 2019-09-24 cs.SD cs.CV cs.LG eess.AS 81%

CochleaNet: A Robust Language-independent Audio-Visual Model for Speech Enhancement

Mandar Gogate, Kia Dashtipour, Ahsan Adeel, Amir Hussain

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 34 pages, 11 figures, Submitted to Information Fusion

1907.04975 2019-07-12 cs.CV cs.SD eess.AS 81%

My lips are concealed: Audio-visual speech enhancement through obstructions

Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to Interspeech 2019

1907.04926 2019-07-12 eess.AS cs.MM cs.SD eess.IV 81%

Synchronizing Audio-Visual Film Stimuli in Unity (version 5.5.1f1): Game Engines as a Tool for Research

Javier Sanz, Andreas Wulff-Abramsson, Carlos Aguilar-Paredes, Luis Emilio Bruni, Lydia Sanchez

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.MM、eess.AS

Comments 13 Pages

1906.02112 2019-07-10 eess.AS cs.CV eess.IV 81%

Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition

Pingchuan Ma, Stavros Petridis, Maja Pantic

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted for publication at Interspeech 2019

1906.10042 2019-06-25 cs.SD cs.CV eess.AS 81%

Who said that?: Audio-visual speaker diarisation of real-world meetings

Joon Son Chung, Bong-Jin Lee, Icksang Han

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to Interspeech 2019

1904.09013 2019-04-22 cs.CV cs.SD eess.AS eess.IV 81%

Self-Supervised Audio-Visual Co-Segmentation

Andrew Rouditchenko, Hang Zhao, Chuang Gan, Josh McDermott, Antonio Torralba

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICASSP 2019

1805.06606 2019-03-11 cs.CL cs.AI cs.HC 81%

Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data

Chan Woo Lee, Kyu Ye Song, Jihoon Jeong, Woo Yong Choi

专题命中音频语音多模态：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Inaccurate scientific facts listed in document

1902.05347 2019-02-15 cs.MM cs.IR cs.SD eess.AS 81%

Multimodal music information processing and retrieval: survey and future challenges

Federico Simonetta, Stavros Ntalampiras, Federico Avanzini

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.MM、eess.AS

Journal ref in 2019 International Workshop on Multilayer Music Representation and Processing, Milano, IEEE 2019

1812.06071 2018-12-17 cs.CV cs.LG cs.MM 81%

On Attention Modules for Audio-Visual Synchronization

Naji Khosravan, Shervin Ardeshir, Rohit Puri

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、cs.MM

1811.10813 2018-11-28 cs.CV eess.AS 81%

Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion

Suwon Shon, Tae-Hyun Oh, James Glass

专题命中音频语音多模态：audio-visual(title)；multi-modal(abstract)；分类 cs.CV、eess.AS

1804.03641 2018-10-10 cs.CV cs.SD eess.AS 81%

Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

Andrew Owens, Alexei A. Efros

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

1809.00758 2018-10-03 cs.LG cs.CV cs.SD eess.AS stat.ML 81%

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Myungsu Chae, Tae-Ho Kim, Young Hoon Shin, June-Woo Kim, Soo-Young Lee

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV、eess.AS

Comments IROS 2018 Workshop on Crossmodal Learning for Intelligent Robotics

1805.07467 2018-09-24 cs.CL cs.SD eess.AS 81%

Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces

Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to NIPS 2018. v2 added the majority word baseline results and other minor fixes. arXiv admin note: text overlap with arXiv:1710.04087 by other authors

1804.03619 2018-08-13 cs.SD cs.CV eess.AS 81%

Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation

Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T. Freeman, Michael Rubinstein

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to SIGGRAPH 2018. Project webpage: https://looking-to-listen.github.io

Journal ref ACM Trans. Graph. 37(4): 112:1-112:11 (2018)

1803.10299 2018-06-19 cs.CL cs.SD eess.AS 81%

Multi-Modal Data Augmentation for End-to-End ASR

Adithya Renduchintala, Shuoyang Ding, Matthew Wiesner, Shinji Watanabe

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 5 Pages, 1 Figure, accepted at INTERSPEECH 2018

1802.05521 2018-02-16 cs.CV cs.SD eess.AS 81%

Deep Learning for Lip Reading using Audio-Visual Information for Urdu Language

M Faisal, Sanaullah Manzoor

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV、eess.AS

1511.04401 2017-12-08 cs.CV cs.CL cs.LG cs.NE 81%

Symbol Grounding Association in Multimodal Sequences with Missing Elements

Federico Raue, Andreas Dengel, Thomas M. Breuel, Marcus Liwicki

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Under review on Journal of Artificial Intelligence Research (JAIR) -- Special Track on Deep Learning, Knowledge Representation, and Reasoning

1707.09538 2017-08-01 cs.MM cs.CL 81%

Benchmarking Multimodal Sentiment Analysis

Erik Cambria, Devamanyu Hazarika, Soujanya Poria, Amir Hussain, R. B. V. Subramaanyam

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted in CICLing 2017

1703.01725 2017-03-07 cs.SI cs.CL cs.CV physics.soc-ph 81%

Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity

Jack Hessel, Lillian Lee, David Mimno

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 10 pages, data and models available at http://www.cs.cornell.edu/~jhessel/cats/cats.html, Proceedings of WWW 2017

1702.06151 2017-02-22 cs.CV cs.IR cs.LG cs.MM 81%

Developing a comprehensive framework for multimodal feature extraction

Quinten McNamara, Alejandro de la Vega, Tal Yarkoni

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV、cs.MM

1606.06259 2016-11-18 cs.CL cs.MM 81%

MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

Amir Zadeh, Rowan Zellers, Eli Pincus, Louis-Philippe Morency

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted as Journal Publication in IEEE Intelligent Systems

Journal ref IEEE Intelligent Systems 31.6 (2016): 82-88

cs/0505064 2016-11-17 cs.HC cs.AI cs.CV cs.LG cs.RO 81%

Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

P. C. McGuire, J. Fritsch, J. J. Steil, F. Roethling, G. A. Fink, S. Wachsmuth, G. Sagerer, H. Ritter

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 8 figures

Journal ref Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland, IEEE publications, pp. 1082-1089 (2002)

2508.00265 2025-08-06 cs.CV 80%

Multimodal Referring Segmentation: A Survey

Henghui Ding, Song Tang, Shuting He, Chang Liu, Zuxuan Wu, Yu-Gang Jiang

机构 * Fudan University（复旦大学）； Shanghai University of Finance and Economics（上海财经大学）； ByteDance Inc（字节跳动公司）

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

Comments Project Page: https://github.com/henghuiding/Awesome-Multimodal-Referring-Segmentation

2405.16464 2024-05-28 cs.RO cs.CV 80%

Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge

Tianchen Deng, Yi Zhou, Wenhua Wu, Mingrui Li, Jingwei Huang, Shuhong Liu, Yanzeng Song, Hao Zuo, Yanbo Wang, Yutao Yue, Hesheng Wang, Weidong Chen

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024 workshop. The 1st winning model in CVPR 2024 UG2+ challenge. The code and configuration of our method are available at https://github.com/dtc111111/Multi-Modal-UAV

2211.03539 2022-11-08 cs.HC cs.AI 80%

Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces

Amr Gomaa

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Journal ref In Proceedings of the 2022 International Conference on Multimodal Interaction, pp. 690-695. 2022

1. 音频语音多模态 4557 篇

An Analysis of Speech Enhancement and Recognition Losses in Limited Resources Multi-talker Single Channel Audio-Visual ASR

SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild

Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers

Exploring Hate Speech Detection in Multimodal Publications

A Multi-Modal Feature Embedding Approach to Diagnose Alzheimer Disease from Spoken Language

CochleaNet: A Robust Language-independent Audio-Visual Model for Speech Enhancement

My lips are concealed: Audio-visual speech enhancement through obstructions

Synchronizing Audio-Visual Film Stimuli in Unity (version 5.5.1f1): Game Engines as a Tool for Research

Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition

Who said that?: Audio-visual speaker diarisation of real-world meetings

Self-Supervised Audio-Visual Co-Segmentation

Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data

Multimodal music information processing and retrieval: survey and future challenges

On Attention Modules for Audio-Visual Synchronization

Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion

Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces

Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation

Multi-Modal Data Augmentation for End-to-End ASR

Deep Learning for Lip Reading using Audio-Visual Information for Urdu Language

Symbol Grounding Association in Multimodal Sequences with Missing Elements

Benchmarking Multimodal Sentiment Analysis

Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity

Developing a comprehensive framework for multimodal feature extraction

MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

Multimodal Referring Segmentation: A Survey

Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge

Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces