多模态大模型

Mixed Signals: Understanding Model Disagreement in Multimodal Empathy Detection

Maya Srikanth, Run Chen, Julia Hirschberg

机构 * Columbia University（哥伦比亚大学） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments To appear in Findings of IJCNLP-AACL 2025

Aligning Brain Signals with Multimodal Speech and Vision Embeddings

Kateryna Shapovalenko, Quentin Auster

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models

Mayank Saini, Arit Kumar Bishwas

机构 * PwC US（普华永道美国） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 15 pages, 4 figures

Socially Aware Music Recommendation: A Multi-Modal Graph Neural Networks for Collaborative Music Consumption and Community-Based Engagement

Kajwan Ziaoddini

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.MM

Shared Latent Representation for Joint Text-to-Audio-Visual Synthesis

Dogucan Yaman, Seymanur Akti, Fevziye Irem Eyiokur, Alexander Waibel

机构 * Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks

Xu Zheng, Zihao Dongfang, Lutao Jiang, Boyuan Zheng, Yulong Guo, Zhenquan Zhang, Giuliano Albanese, Runyi Yang, Mengjiao Ma, Zixin Zhang, Chenfei Liao, Dingcheng Zhen, Yuanhuiyi Lyu, Yuqian Fu, Bin Ren, Linfeng Zhang, Danda Pani Paudel, Nicu Sebe, Luc Van Gool, Xuming Hu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy?

Yiwen Guan, Viet Anh Trinh, Vivek Voleti, Jacob Whitehill

专题命中 音频语音多模态 ：multi-modal(title)；分类 cs.CV、cs.CL、cs.MM

Journal ref IEEE International Conference on Multimedia and Expo (ICME), Nantes, France, 2025, pp. 1-6

Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations

Ahmad Ghannam, Naif Alharthi, Faris Alasmary, Kholood Al Tabash, Shouq Sadah, Lahouari Ghouti

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving

Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca

机构 * Microsoft Azure Research（微软Azure研究） ； Microsoft Research India（微软印度研究） ； University of Virginia（弗吉尼亚大学） ； Microsoft M365 Research（微软M365研究） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Published at ACM SoCC 2025; 14 pages, 20 figures

Audio-Visual Speech Enhancement for Spatial Audio - Spatial-VisualVoice and the MAVE Database

Danielle Yaffe, Ferdinand Campe, Prachi Sharma, Dorothea Kolossa, Boaz Rafaely

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

Joshua Wolfe Brook, Ilia Markov

机构 * Computational Linguistics \& Text Mining Lab Vrije Universiteit Amsterdam 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 8 pages, 9 figures, submitted to LREC 2026

Towards Multimodal Query-Based Spatial Audio Source Extraction

Chenxin Yu, Hao Ma, Xu Li, Xiao-Lei Zhang, Mingjie Shao, Chi Zhang, Xuelong Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Submitted to ICASSP 2026

Adaptive vector steering: A training-free, layer-wise intervention for hallucination mitigation in large audio and multimodal models

Tsung-En Lin, Kuan-Yi Lee, Hung-Yi Lee

机构 * National Taiwan University（国立台湾大学） ； ASUS Open Cloud Infrastructure Software Center（ASUS开放云基础设施软件中心） 纠错

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 eess.AS

Comments Note: This preprint is a version of the paper submitted to ICASSP 2026. The author list here includes contributors who provided additional supervision and guidance. The official ICASSP submission may differ slightly in author composition

A Fast and Lightweight Model for Causal Audio-Visual Speech Separation

Wendi Sang, Kai Li, Runxuan Yang, Jianqiang Huang, Xiaolin Hu

机构 * School of Computer Technology and Application（计算机技术与应用学院） ； Intelligent Computing and Application Laboratory of Qinghai Province（青海省智能计算与应用实验室） ； Department of Computer Science and Technology（计算机科学与技术系） ； Institute for AI（人工智能研究院） ； Tsinghua Laboratory of Brain and Intelligence (THBI)（清华大学脑与智能实验室） ； IDG/McGovern Institute for Brain Research（IDG/麦克戈维脑研究学院） ； Tsinghua University（清华大学） ； Chinese Institute for Brain Research (CIBR)（中国脑科学研究院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted by ECAI 2025

Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning

Xinying Hou, Ruiwei Xiao, Runlong Ye, Michael Liut, John Stamper

机构 * University of Michigan（密歇根大学） ； Carnegie Mellon University（卡内基梅隆大学） ； University of Toronto（多伦多大学） ； University of Toronto Mississauga（多伦多大学滑铁卢分校） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 7 pages, accepted to SIGCSE2026

Leveraging Self-Supervised Audio-Visual Pretrained Models to Improve Vocoded Speech Intelligibility in Cochlear Implant Simulation

Richard Lee Lai, Jen-Cheng Hou, I-Chun Chern, Kuo-Hsuan Hung, Yi-Ting Chen, Mandar Gogate, Tughrul Arslan, Amir Hussain, Yu Tsao

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Leveraging Mamba with Full-Face Vision for Audio-Visual Speech Enhancement

Rong Chao, Wenze Ren, You-Jin Li, Kuo-Hsuan Hung, Sung-Feng Huang, Szu-Wei Fu, Wen-Huang Cheng, Yu Tsao

机构 * Academia Sinica（台湾“中央研究院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted to Interspeech 2025 Workshop

Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments

Hailong Zhang, Yinfeng Yu, Liejun Wang, Fuchun Sun, Wendong Zheng

机构 * Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术研究中心） ； School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院） ； Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； School of Electrical Engineering and Automation, Tianjin University of Technology（天津理工大学电气工程与自动化学院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Comments Main paper (15 pages). Accepted for publication by ICONIP( International Conference on Neural Information Processing) 2025

Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models

Zhen Xiong, Yujun Cai, Zhecheng Li, Junsong Yuan, Yiwei Wang

机构 * University of Southern California（南加州大学） ； University of Queensland（昆士兰大学） ； University of California, San Diego（加州大学圣地亚哥分校） ； University of Buffalo（布法罗大学） ； University of California, Merced（加州大学默塞德分校） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol

Misk Al Zahidy, Kerly Guevara Maldonado, Luis Vilatuna Andrango, Ana Cristina Proano, Ana Gabriela Claros, Maria Lizarazo Jimenez, David Toro-Tobon, Victor M. Montori, Oscar J. Ponce-Ponte, Juan P. Brito

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 23 pages, 2 figures, 2 tables

Real-Time System for Audio-Visual Target Speech Enhancement

T. Aleksandra Ma, Sile Yin, Li-Chia Yang, Shuo Zhang

机构 * Bose Corporation（博世公司） ； Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted into WASPAA 2025 demo session

PoolingVQ: A VQVAE Variant for Reducing Audio Redundancy and Boosting Multi-Modal Fusion in Music Emotion Analysis

Dinghao Zou, Yicheng Gong, Xiaokang Li, Xin Cao, Sunbowen Lee

机构 * Wuhan University of Science and Technology（武汉科技大学） 纠错

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 eess.AS

The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio

Renhao Wang, Haoran Geng, Tingle Li, Feishi Wang, Gopala Anumanchipalli, Trevor Darrell, Boyi Li, Pieter Abbeel, Jitendra Malik, Alexei A. Efros

机构 * University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Conference on Robot Learning 2025

Interpreting the Role of Visemes in Audio-Visual Speech Recognition

Aristeidis Papadopoulos, Naomi Harte

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted into Automatic Speech Recognition and Understanding- ASRU 2025

Music4All A+A: A Multimodal Dataset for Music Information Retrieval Tasks

Jonas Geiger, Marta Moscati, Shah Nawaz, Markus Schedl

机构 * Johannes Kepler University Linz（约翰内斯·开普勒大学林茨） ； Human-centered AI Group, AI Lab, Linz Institute of Technology（以人为本的人工智能小组、人工智能实验室、林茨技术研究所） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments 7 pages, 6 tables, IEEE International Conference on Content-Based Multimedia Indexing (IEEE CBMI)

Diffusion-Based Unsupervised Audio-Visual Speech Separation in Noisy Environments with Noise Prior

Yochai Yemini, Rami Ben-Ari, Sharon Gannot, Ethan Fetaya

机构 * Faculty of Engineering, Bar-Ilan University（巴伊兰大学工程学院） ； OriginAI 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

MMORE: Massive Multimodal Open RAG & Extraction

Alexandre Sallinen, Stefan Krsteski, Paul Teiletche, Marc-Antoine Allard, Baptiste Lecoeur, Michael Zhang, Fabrice Nemo, David Kalajdzic, Matthias Meyer, Mary-Anne Hartley

机构 * École Polytechnique Fédérale de Lausanne (EPFL), Switzerland（瑞士联邦理工学院洛桑校区） ； ETH Zürich, Switzerland（瑞士苏黎世联邦理工学院） ； T.H. Chan School of Public Health, Harvard University, USA（哈佛大学T.H. Chan公共卫生学院） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments This paper was originally submitted to the CODEML workshop for ICML 2025. 9 pages (including references and appendices)

WeaveMuse: An Open Agentic System for Multimodal Music Understanding and Generation

Emmanouil Karystinaios

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Accepted at Large Language Models for Music & Audio Workshop (LLM4MA) 2025

MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind

Zheng Zhang, Nuoqian Xiao, Qi Chai, Deheng Ye, Hao Wang

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Tencent（腾讯） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by ACMMM 2025

WAVE-DETR Multi-Modal Visible and Acoustic Real-Life Drone Detector

Razvan Stefanescu, Ethan Oh, Ruben Vazquez, Chris Mesterharm, Constantin Serban, Ritu Chadha

机构 * Peraton Labs（珀顿实验室） 纠错

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 11 figures