多模态大模型

Reinforced Label Denoising for Weakly-Supervised Audio-Visual Video Parsing

Yongbiao Gao, Xiangcheng Sun, Guohua Lv, Deng Yu, Sijiu Niu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

SilVar: Speech Driven Multimodal Model for Reasoning Visual Question Answering and Object Localization

Tan-Hanh Pham, Hoang-Nam Le, Phu-Vinh Nguyen, Chris Ngo, Truong-Son Hy

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages

Deep Speech Synthesis from Multimodal Articulatory Representations

Peter Wu, Bohan Yu, Kevin Scheck, Alan W Black, Aditi S. Krishnapriyan, Irene Y. Chen, Tanja Schultz, Shinji Watanabe, Gopala K. Anumanchipalli

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

AMuSeD: An Attentive Deep Neural Network for Multimodal Sarcasm Detection Incorporating Bi-modal Data Augmentation

Xiyuan Gao, Shubhi Bansal, Kushaan Gowda, Zhu Li, Shekhar Nayak, Nagendra Kumar, Matt Coler

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments This is a preprint version of the paper, submitted and under review at the IEEE Transactions on Affective Computing

Multimodal Sentiment Analysis based on Video and Audio Inputs

Antonio Fernandez, Suzan Awinat

专题命中 音频语音多模态 ：multimodal(title)；分类 cs.CV、cs.AI、cs.MM

Comments Presented as a full paper in the 15th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN 2024) October 28-30, 2024, Leuven, Belgium

Journal ref Procedia Computer Science, Volume 251, 2024, Pages 41-48, ISSN 1877-0509

TECO: Improving Multimodal Intent Recognition with Text Enhancement through Commonsense Knowledge Extraction

Quynh-Mai Thi Nguyen, Lan-Nhi Thi Nguyen, Cam-Van Thi Nguyen

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at PACLIC 2024

SAVEn-Vid: Synergistic Audio-Visual Integration for Enhanced Understanding in Long Video Context

Jungang Li, Sicheng Tao, Yibo Yan, Xiaojie Gu, Haodong Xu, Xu Zheng, Yuanhuiyi Lyu, Linfeng Zhang, Xuming Hu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments The publication has some processing errors (language short-cuts in synthetic data are not avoided) that invalidate some of the conclusions

Text Is Not All You Need: Multimodal Prompting Helps LLMs Understand Humor

Ashwin Baluja

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Generative AI in Multimodal User Interfaces: Trends, Challenges, and Cross-Platform Adaptability

J. Bieniek, M. Rahouti, D. C. Verma

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 13 pages, 4 figures

Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR

Minghan Wang, Yuxia Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024 Findings

Scaling Law Hypothesis for Multimodal Model

Qingyun Sun, Zhen Guo, PIN AI Team

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Efficient Audio-Visual Fusion for Video Classification

Mahrukh Awan, Asmar Nadeem, Armin Mustafa

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments CVMP Short Paper

Multimodal Semantic Communication for Generative Audio-Driven Video Conferencing

Haonan Tong, Haopeng Li, Hongyang Du, Zhaohui Yang, Changchuan Yin, Dusit Niyato

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments accepted by IEEE Wireless Communications Letters

Joint Audio-Visual Idling Vehicle Detection with Streamlined Input Dependencies

Xiwen Li, Rehman Mohammed, Tristalee Mangin, Surojit Saha, Ross T Whitaker, Kerry E. Kelly, Tolga Tasdizen

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

NeuGPT: Unified multi-modal Neural GPT

Yiqian Yang, Yiqun Duan, Hyejeong Jo, Qiang Zhang, Renjing Xu, Oiwi Parker Jones, Xuming Hu, Chin-teng Lin, Hui Xiong

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL

Estuary: A Framework For Building Multimodal Low-Latency Real-Time Socially Interactive Agents

Spencer Lin, Basem Rizk, Miru Jun, Andy Artze, Caitlin Sullivan, Sharon Mozgai, Scott Fisher

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments To be published in ACM Intelligent Virtual Agents (IVA) 2024 [DOI: 10.1145/3652988.3696198] [ACM ISBN: 979-8-4007-0625-7/24/09]

A Survey of Multimodal Sarcasm Detection

Shafkat Farabi, Tharindu Ranasinghe, Diptesh Kanojia, Yu Kong, Marcos Zampieri

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Published in the Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence Survey Track. Pages 8020-8028

Towards Multimodal Emotional Support Conversation Systems

Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, Richang Hong

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments This work has been submitted to the IEEE for possible publication

With Ears to See and Eyes to Hear: Sound Symbolism Experiments with Multimodal Large Language Models

Tyler Loakman, Yucheng Li, Chenghua Lin

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024 (Camera Ready)

A Pattern to Align Them All: Integrating Different Modalities to Define Multi-Modal Entities

Gianluca Apriceno, Valentina Tamma, Tania Bailoni, Jacopo de Berardinis, Mauro Dragoni

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments 20 pages, 6 figures

MMHead: Towards Fine-grained Multi-modal 3D Facial Animation

Sijing Wu, Yunhao Li, Yichao Yan, Huiyu Duan, Ziwei Liu, Guangtao Zhai

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by ACMMM 2024. Project page: https://wsj-sjtu.github.io/MMHead/

Gumbel Rao Monte Carlo based Bi-Modal Neural Architecture Search for Audio-Visual Deepfake Detection

Aravinda Reddy PN, Raghavendra Ramachandra, Krothapalli Sreenivasa Rao, Pabitra Mitra Vinod Rathod

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 eess.AS

Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond

Soyeon Caren Han, Feiqi Cao, Josiah Poon, Roberto Navigli

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at ACM-MM 2024

MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models

Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, Xuming Hu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Identifiable Shared Component Analysis of Unpaired Multimodal Mixtures

Subash Timilsina, Sagar Shrestha, Xiao Fu

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Robust Audio-Visual Speech Enhancement: Correcting Misassignments in Complex Environments with Advanced Post-Processing

Wenze Ren, Kuo-Hsuan Hung, Rong Chao, YouJin Li, Hsin-Min Wang, Yu Tsao

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments The 27th International Conference of the Oriental COCOSDA

StyleFusion TTS: Multimodal Style-control and Enhanced Feature Fusion for Zero-shot Text-to-speech Synthesis

Zhiyong Chen, Xinnuo Li, Zhiqi Ai, Shugong Xu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments The 7th Chinese Conference on Pattern Recognition and Computer Vision PRCV 2024

Journal ref The 7th Chinese Conference on Pattern Recognition and Computer Vision PRCV 2024

A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answering

Georgios Sidiropoulos, Evangelos Kanoulas

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework

Anusha Chhabra, Dinesh Kumar Vishwakarma

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

MRAC Track 1: 2nd Workshop on Multimodal, Generative and Responsible Affective Computing

Shreya Ghosh, Zhixi Cai, Abhinav Dhall, Dimitrios Kollias, Roland Goecke, Tom Gedeon

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments ACM MM Workshop 2024. Workshop webpage: https://react-ws.github.io/2024/