多模态信息融合

Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling

Haogeng Liu, Qihang Fan, Tingkai Liu, Linjie Yang, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Audio Visual Speaker Localization from EgoCentric Views

Jinzheng Zhao, Yong Xu, Xinyuan Qian, Wenwu Wang

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.MM

VideoAdviser: Video Knowledge Distillation for Multimodal Transfer Learning

Yanan Wang, Donghuo Zeng, Shinya Wada, Satoshi Kurihara

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by IEEE Access

Journal ref in IEEE Access, vol. 11, pp. 51229-51240, 2023

TriGait: Aligning and Fusing Skeleton and Silhouette Gait Data via a Tri-Branch Network

Yan Sun, Xueling Feng, Liyan Ma, Long Hu, Mark Nixon

专题命中 音视频/视觉语言融合 ：hybrid fusion(abstract)；分类 cs.CV

Comments Accepted by IJCB 2023

TACOformer:Token-channel compounded Cross Attention for Multimodal Emotion Recognition

Xinda Li

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.MM

Comments Accepted by IJCAI 2023- AI4TS workshop

Single Frame Semantic Segmentation Using Multi-Modal Spherical Images

Suresh Guttikonda, Jason Rambach

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted at WACV 2024

3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment

Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, Qing Li

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning

Xiaobao Guo, Nithish Muthuchamy Selvaraj, Zitong Yu, Adams Wai-Kin Kong, Bingquan Shen, Alex Kot

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.CV

Comments 11 pages, 6 figures

Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

Aihua Zheng, Zhiqi Ma, Zi Wang, Chenglong Li

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Provable Dynamic Fusion for Low-Quality Multimodal Data

Qingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu, Huazhu Fu, Joey Tianyi Zhou, Xi Peng

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by ICML 2023

ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

Peng Wang, Shijie Wang, Junyang Lin, Shuai Bai, Xiaohuan Zhou, Jingren Zhou, Xinggang Wang, Chang Zhou

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments 30 pages, 9 figures, 18 tables

FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofing

Ajian Liu, Zichang Tan, Zitong Yu, Chenxu Zhao, Jun Wan, Yanyan Liang, Zhen Lei, Du Zhang, Stan Z. Li, Guodong Guo

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection

Linmei Hu, Ziwang Zhao, Weijian Qi, Xuemeng Song, Liqiang Nie

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.MM

Husformer: A Multi-Modal Transformer for Multi-Modal Human State Recognition

Ruiqi Wang, Wonse Jo, Dezhong Zhao, Weizheng Wang, Baijian Yang, Guohua Chen, Byung-Cheol Min

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.RO

Self-supervised Contrastive Learning for Audio-Visual Action Recognition

Yang Liu, Ying Tan, Haoyuan Lan

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments 6 pages, 2 figures

MMCosine: Multi-Modal Cosine Loss Towards Balanced Audio-Visual Fine-Grained Learning

Ruize Xu, Ruoxuan Feng, Shi-Xiong Zhang, Di Hu

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.MM

Delivering Arbitrary-Modal Semantic Segmentation

Jiaming Zhang, Ruiping Liu, Hao Shi, Kailun Yang, Simon Reiß, Kunyu Peng, Haodong Fu, Kaiwei Wang, Rainer Stiefelhagen

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023. Dataset and our code are at: https://jamycheung.github.io/DELIVER.html

Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics

Surbhi Madan, Monika Gahalawat, Tanaya Guha, Roland Goecke, Ramanathan Subramanian

专题命中 音视频/视觉语言融合 ：feature-level fusion(abstract)；分类 cs.CV

Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis

Haoran Sun, Yang Wang, Haipeng Liu, Biao Qian

专题命中 音视频/视觉语言融合 ：image fusion(abstract)；分类 cs.CV

Comments 13 pages, 8 figures, accepted by Chinese Journal of Electronics

Flexible-modal Deception Detection with Audio-Visual Adapter

Zhaoxu Li, Zitong Yu, Nithish Muthuchamy Selvaraj, Xiaobao Guo, Bingquan Shen, Adams Wai-Kin Kong, Alex Kot

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Improving the Modality Representation with Multi-View Contrastive Learning for Multimodal Sentiment Analysis

Peipei Liu, Xin Zheng, Hong Li, Jie Liu, Yimo Ren, Hongsong Zhu, Limin Sun

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.MM

Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models

Lei Wang, Jiabang He, Xing Xu, Ning Liu, Hui Liu

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by AAAI 2023. Code is available at https://github.com/MAEHCM/AET

Panoramic Video Salient Object Detection with Ambisonic Audio Guidance

Xiang Li, Haoyuan Cao, Shijie Zhao, Junlin Li, Li Zhang, Bhiksha Raj

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Comments NeurIPS 2022. Project Website: https://ashkamath.github.io/FIBER_page

Video-based Cross-modal Auxiliary Network for Multimodal Sentiment Analysis

Rongfei Chen, Wenju Zhou, Yang Li, Huiyu Zhou

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV

Language Guided Networks for Cross-modal Moment Retrieval

Kun Liu, Huadong Ma, Chuang Gan

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comprehensive Multi-Modal Interactions for Referring Image Segmentation

Kanishk Jain, Vineet Gandhi

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments Findings of ACL 2022

Journal ref 2022.findings-acl.270

A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!

Chenglizhao Chen, Mengke Song, Wenfeng Song, Li Guo, Muwei Jian

专题命中 音视频/视觉语言融合 ：audio-visual fusion(abstract)；分类 cs.CV

TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer

Jiajun Deng, Zhengyuan Yang, Daqing Liu, Tianlang Chen, Wengang Zhou, Yanyong Zhang, Houqiang Li, Wanli Ouyang

专题命中 音视频/视觉语言融合 ：multi-modal fusion(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2104.08541

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

专题命中 音视频/视觉语言融合 ：multimodal fusion(abstract)；分类 cs.CV