多模态信息融合

MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion

Chuang Ma, Yu Pei, Jianhang Zhang, Shaokai Zhao, Bowen Ji, Liang Xie, Ye Yan, Erwei Yin

机构 * The Defense Innovation Institute, Academy of Military Sciences（国防科技创新院，军事科学学院） ； Northwestern Polytechnical University（西北工业大学） 纠错

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancement

Teng Hu, Zhentao Yu, Zhengguang Zhou, Jiangning Zhang, Yuan Zhou, Qinglin Lu, Ran Yi

机构 * Shanghai Jiao Tong University（上海交通大学） ； Tencent Hunyuan（腾讯文英） ； Zhejiang University（浙江大学） 纠错

专题命中 音视频/视觉语言融合 ：image fusion(abstract)；分类 cs.CV

Zero Shot Composed Image Retrieval

Santhosh Kakarla, Gautama Shastry Bulusu Venkata

机构 * George Mason University（乔治·玛森大学） 纠错

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments 8 pages, 3 figures

SAM2-LOVE: Segment Anything Model 2 in Language-aided Audio-Visual Scenes

Yuji Wang, Haoran Xu, Yong Liu, Jiaze Li, Yansong Tang

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学） 纠错

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments CVPR 2025

AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection

Peng Wu, Wanshun Su, Guansong Pang, Yujia Sun, Qingsen Yan, Peng Wang, Yanning Zhang

机构 * School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机科学学院） ； School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算与信息系统学院） ； School of Artifical Intelligence, Xidian University（西安电子科技大学人工智能学院） 纠错

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.CV

Comments 12 pages, 6 figures, 9 tables. This work has been submitted to the IEEE for possible publication

Cognitive Disentanglement for Referring Multi-Object Tracking

Shaofeng Liang, Runwei Guan, Wangwang Lian, Daizong Liu, Xiaolou Sun, Dongming Wu, Yutao Yue, Weiping Ding, Hui Xiong

机构 * Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China)（青岛软件研究所，计算机科学与技术学院，中国石油大学（华东）） ； Thrust of AI, Hong Kong University of Science and Technology (GuangZhou)（人工智能研究组，香港科学与技术大学（广州）） ； Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software（山东省智能油气工业软件重点实验室） ； Wangxuan Institute of Computer Technology, Peking University（王轩计算机技术研究所，北京大学） ； School of Automation, Southeast University（自动化学院，东南大学） ； School of Computer Science, Beijing Institute of Technology（计算机科学学院，北京理工大学） ； School of Artificial Intelligence and Computer Science, Nantong University（人工智能与计算机科学学院，南通大学） ； Thrust of Intelligent Transportation, Hong Kong University of Science and Technology (GuangZhou)（智能交通研究组，香港科学与技术大学（广州）） ； Department of Electrical Engineering and Electronics, University of Liverpool（电子工程与电子学院，利物浦大学） 纠错

专题命中 音视频/视觉语言融合 ：information fusion(abstract)；分类 cs.CV

Comments 27 pages, 12 figures

VISTA: Enhancing Vision-Text Alignment in MLLMs via Cross-Modal Mutual Information Maximization

Mingxiao Li, Na Su, Fang Qu, Zhizhou Zhong, Ziyang Chen, Yuan Li, Zhaopeng Tu, Xiaolong Li

机构 * Tencent（腾讯） 纠错

专题命中 音视频/视觉语言融合 ：information fusion(abstract)；分类 cs.CV

TS-VLM: Text-Guided SoftSort Pooling for Vision-Language Models in Multi-View Driving Reasoning

Lihong Chen, Hossein Hassani, Soodeh Nikan

机构 * Department of Electrical and Computer Engineering, Western University（电气与计算机工程系，西方大学） 纠错

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation

Sungnyun Kim, Sungwoo Cho, Sangmin Bae, Kangwook Jang, Se-Young Yun

机构 * Kim Jaechul Graduate School of AI, KAIST（金jaechul人工智能研究生院，韩国科学技术院） ； School of Electrical Engineering, KAIST（电气工程学院，韩国科学技术院） 纠错

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.MM

Comments ICLR 2025; 22 pages, 6 figures, 14 tables

Adversarial Attack for RGB-Event based Visual Object Tracking

Qiang Chen, Xiao Wang, Haowen Wang, Bo Jiang, Lin Zhu, Dawei Zhang, Yonghong Tian, Jin Tang

机构 * School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院） ； Beijing Institute of Technology（北京理工大学） ； Zhejiang Normal University（浙江师范大学） ； Peng Cheng Laboratory（鹏城实验室） ； National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学多媒体信息处理国家重点实验室） ； School of Electronic and Computer Engineering, Shenzhen, Graduate School, Peking University（北京大学深圳研究生院电子与计算机工程学院） 纠错

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework

Wentao Wu, Xiao Wang, Chenglong Li, Bo Jiang, Jin Tang, Bin Luo, Qi Liu

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

DTFSal: Audio-Visual Dynamic Token Fusion for Video Saliency Prediction

Kiana Hooshanfar, Alireza Hosseini, Ahmad Kalhor, Babak Nadjar Araabi

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions

Jinzheng Zhao, Yong Xu, Xinyuan Qian, Davide Berghi, Peipei Wu, Meng Cui, Jianyuan Sun, Philip J. B. Jackson, Wenwu Wang

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.MM

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models

Kim Sung-Bin, Jeongsoo Choi, Puyuan Peng, Joon Son Chung, Tae-Hyun Oh, David Harwath

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.CV

Comments https://voicecraft-dub.github.io/

COST: Contrastive One-Stage Transformer for Vision-Language Small Object Tracking

Chunhui Zhang, Li Liu, Jialin Gao, Xin Sun, Hao Wen, Xi Zhou, Shiming Ge, Yanfeng Wang

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments Preprint submitted to Elsevier. https://github.com/983632847/Awesome-Multimodal-Object-Tracking

CoLLM: A Large Language Model for Composed Image Retrieval

Chuong Huynh, Jinyu Yang, Ashish Tawari, Mubarak Shah, Son Tran, Raffay Hamid, Trishul Chilimbi, Abhinav Shrivastava

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments CVPR 2025. Project page: https://collm-cvpr25.github.io/

Dual-Domain Homogeneous Fusion with Cross-Modal Mamba and Progressive Decoder for 3D Object Detection

Xuzhong Hu, Zaipeng Duan, Pei An, Jun zhang, Jie Ma

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments 13 pages, 9 figures

LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport

Kyeongha Rho, Hyeongkeun Lee, Valentio Iverson, Joon Son Chung

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.MM

Comments 5 pages, 2 figures; Accepted to ICASSP 2025

TI-JEPA: An Innovative Energy-based Joint Embedding Strategy for Text-Image Multimodal Systems

Khang H. N. Vo, Duc P. T. Nguyen, Thong Nguyen, Tho T. Quan

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Recognition of Dysarthria in Amyotrophic Lateral Sclerosis patients using Hypernetworks

Loukas Ilias, Dimitris Askounis

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment

Chunhui Zhang, Xin Sun, Yiqian Yang, Li Liu, Qiong Liu, Xi Zhou, Yanfeng Wang

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments In this version, we corrected some typos

Enhancing Audio-Visual Spiking Neural Networks through Semantic-Alignment and Cross-Modal Residual Learning

Xiang He, Dongcheng Zhao, Yiting Dong, Guobin Shen, Xin Yang, Yi Zeng

专题命中 音视频/视觉语言融合 ：information fusion(abstract)；分类 cs.CV

Comments The manuscript is under review and the code is available https://github.com/Brain-Cog-Lab/S-CMRL

Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalities

Yidi Li, Yihan Li, Yixin Guo, Bin Ren, Zhenhuan Xu, Hao Guo, Hong Liu, Nicu Sebe

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments We request to withdraw our paper from arXiv due to unresolved author disagreements about the data interpretation and study conclusions. To maintain scientific integrity, we believe withdrawing the paper is necessary. We regret any confusion caused

Triple Path Enhanced Neural Architecture Search for Multimodal Fake News Detection

Bo Xu, Qiujie Xie, Jiahui Zhou, Linlin Zong

专题命中 音视频/视觉语言融合 ：information fusion(abstract)；分类 cs.CV

Comments IEEE International Conference on Acoustics, Speech, and Signal Processing(ICASSP 2025)

Audio-Visual Approach For Multimodal Concurrent Speaker Detection

Amit Eliav, Sharon Gannot

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 eess.IV

Comments 12 pages

AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation

Sitong Gong, Yunzhi Zhuge, Lu Zhang, Yifan Wang, Pingping Zhang, Lijun Wang, Huchuan Lu

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted to IEEE Transactions on Multimedia (TMM)

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

Haohe Liu, Gael Le Lan, Xinhao Mei, Zhaoheng Ni, Anurag Kumar, Varun Nagaraja, Wenwu Wang, Mark D. Plumbley, Yangyang Shi, Vikas Chandra

专题命中 音视频/视觉语言融合 ：information fusion(abstract)；分类 cs.MM

AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment

Yan Li, Yifei Xing, Xiangyuan Lan, Xin Li, Haifeng Chen, Dongmei Jiang

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

SimBase: A Simple Baseline for Temporal Video Grounding

Peijun Bao, Alex C. Kot

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments Technical report

Quality-Aware End-to-End Audio-Visual Neural Speaker Diarization

Mao-Kui He, Jun Du, Shu-Tong Niu, Qing-Feng Liu, Chin-Hui Lee

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.MM