多模态信息融合 - arXivDaily 专题

2203.12122 2022-04-22 cs.SD cs.MM eess.AS 57%

On Adversarial Robustness of Large-scale Audio Visual Learning

Juncheng B Li, Shuhui Qu, Xinjian Li, Po-Yao Huang, Florian Metze

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.MM

Journal ref 2022 International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2022)

2203.15969 2022-03-31 cs.CV 57%

Deeply Interleaved Two-Stream Encoder for Referring Video Segmentation

Guang Feng, Lihe Zhang, Zhiwei Hu, Huchuan Lu

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

2202.09263 2022-02-21 cs.LG cs.MM 57%

Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?

Vandana Rajan, Alessio Brutti, Andrea Cavallaro

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.MM

Comments Accepted at ICASSP 2022

2202.07360 2022-02-16 cs.HC cs.CV 57%

Multimodal Driver Referencing: A Comparison of Pointing to Objects Inside and Outside the Vehicle

Abdul Rafey Aftab, Michael von der Beeck

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Journal ref 27th International Conference on Intelligent User Interfaces (IUI '22), March 22--25, 2022, Helsinki, Finland

2201.09828 2022-01-25 cs.LG cs.CV 57%

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis

Georgios Paraskevopoulos, Efthymios Georgiou, Alexandros Potamianos

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted, ICASSP 2022

2112.07423 2021-12-15 cs.CV 57%

Multi-Modal Perception Attention Network with Self-Supervised Learning for Audio-Visual Speaker Tracking

Yidi Li, Hong Liu, Hao Tang

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted by AAAI2022

2108.10509 2021-08-25 cs.MM 57%

Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues

Peng Qi, Juan Cao, Xirong Li, Huan Liu, Qiang Sheng, Xiaoyue Mi, Qin He, Yongbiao Lv, Chenyang Guo, Yingchao Yu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.MM

Comments To appear in MM 2021 industrial track (long paper)

2108.09980 2021-08-24 cs.CV cs.AI cs.LG 57%

TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment

Jianwei Yang, Yonatan Bisk, Jianfeng Gao

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted by ICCV 2021

2007.12519 2021-04-23 cs.CV cs.LG 57%

HEU Emotion: A Large-scale Database for Multi-modal Emotion Recognition in the Wild

Jing Chen, Chenhui Wang, Kejun Wang, Chaoqun Yin, Cong Zhao, Tao Xu, Xinyi Zhang, Ziqiang Huang, Meichen Liu, Tao Yang

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments Neural Comput & Applic (2021)

2012.09408 2021-04-15 eess.AS cs.SD eess.SP 57%

Interactive Speech and Noise Modeling for Speech Enhancement

Chengyu Zheng, Xiulian Peng, Yuan Zhang, Sriram Srinivasan, Yan Lu

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 eess.SP

Comments AAAI 2021 (Accepted)

2010.15251 2021-03-01 cs.CV cs.AI cs.CL cs.LG 57%

Fusion Models for Improved Visual Captioning

Marimuthu Kalimuthu, Aditya Mogadala, Marius Mosbach, Dietrich Klakow

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted at "Multi-Modal Deep Learning: Challenges and Applications" (MMDLCA), International Conference on Pattern Recognition (ICPR)-2020, Milano, Italia

Journal ref Springer LNCS, volume 12666, 2021

2101.12059 2021-02-02 cs.CV cs.CL 57%

VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs

Xudong Lin, Gedas Bertasius, Jue Wang, Shih-Fu Chang, Devi Parikh, Lorenzo Torresani

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Work in progress

2012.14891 2021-01-01 cs.CV cs.LG 57%

Detecting Hate Speech in Multi-modal Memes

Abhishek Das, Japsimar Singh Wahi, Siyao Li

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

2005.00200 2020-10-01 cs.CV cs.CL cs.LG 57%

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by EMNLP 2020

2009.01224 2020-09-03 eess.SP cs.HC 57%

American Sign Language Recognition Using RF Sensing

Sevgi Z. Gurbuz, Ali C. Gurbuz, Evie A. Malaia, Darrin J. Griffin, Chris Crawford, M. Mahbubur Rahman, Emre Kurtoglu, Ridvan Aksu, Trevor Macks, Robiulhossain Mdrafi

专题命中音视频/视觉语言融合：feature-level fusion(abstract)；分类 eess.SP

Comments submitted

Journal ref IEEE Sensors Journal, August 2020

2008.04858 2020-08-31 cs.CV 57%

KBGN: Knowledge-Bridge Graph Network for Adaptive Vision-Text Reasoning in Visual Dialogue

Xiaoze Jiang, Siyi Du, Zengchang Qin, Yajing Sun, Jing Yu

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments Accepted by the 28th ACM International Conference on Multimedia (ACM MM 2020), Oral

2008.06581 2020-08-18 cs.CV cs.SD eess.AS 57%

Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention

Bin Duan, Hao Tang, Wei Wang, Ziliang Zong, Guowei Yang, Yan Yan

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2008.05023 2020-08-13 cs.CV 57%

Audio- and Gaze-driven Facial Animation of Codec Avatars

Alexander Richard, Colin Lea, Shugao Ma, Juergen Gall, Fernando de la Torre, Yaser Sheikh

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2007.04901 2020-07-10 cs.CV 57%

Cross-Modal Weighting Network for RGB-D Salient Object Detection

Gongyang Li, Zhi Liu, Linwei Ye, Yang Wang, Haibin Ling

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments Accepted in ECCV2020. Code: https://github.com/MathLee/CMWNet

2007.04364 2020-07-10 cs.CV cs.AI 57%

Temporal aggregation of audio-visual modalities for emotion recognition

Andreea Birhala, Catalin Nicolae Ristea, Anamaria Radoi, Liviu Cristian Dutu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

1908.08498 2019-08-23 cs.CV 57%

EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted for presentation at ICCV 2019

1906.10096 2019-06-25 cs.CV 57%

Audio-Visual Kinship Verification

Xiaoting Wu, Eric Granger, Xiaoyi Feng

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments 12 pages

1904.13072 2019-05-01 cs.CV 57%

Cross-Modal Message Passing for Two-stream Fusion

Dong Wang, Yuan Yuan, Qi Wang

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments 2018 IEEE International Conference on Acoustics, Speech and Signal Processing

1810.12829 2018-12-26 cs.CV 57%

Cross-Modal Attentional Context Learning for RGB-D Object Detection

Guanbin Li, Yukang Gan, Hejun Wu, Nong Xiao, Liang Lin

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments Accept as a regular paper to IEEE Transactions on Image Processing

1810.10565 2018-10-26 cs.CV cs.AI cs.HC 57%

Multimodal Polynomial Fusion for Detecting Driver Distraction

Yulun Du, Chirag Raman, Alan W Black, Louis-Philippe Morency, Maxine Eskenazi

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments INTERSPEECH 2018

1603.09725 2018-10-15 cs.CV cs.SD 57%

Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion

Israel D. Gebru, Silèye Ba, Xiaofei Li, Radu Horaud

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments 14 pages, 6 figures, 5 tables

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1086 - 1099, 2018

1809.04931 2018-09-14 cs.HC cs.CV cs.LG 57%

Multimodal Local-Global Ranking Fusion for Emotion Recognition

Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments ACM International Conference on Multimodal Interaction (ICMI 2018)

1804.05448 2018-04-17 cs.CL cs.AI cs.CV 57%

Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning

Xin Wang, Yuan-Fang Wang, William Yang Wang

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments NAACL 2018

1803.08842 2018-03-26 cs.CV 57%

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments 23 pages, 7 figures

1408.2700 2016-04-18 cs.SD cs.MM stat.AP stat.ML 57%

Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression

Antoine Deleforge, Radu Horaud, Yoav Schechner, Laurent Girin

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.MM

Comments 15 pages, 8 figures

Journal ref IEEE Transactions on Audio, Speech, and Language Processing 23(4), 718-731, April, 2015

1. 音视频/视觉语言融合 495 篇

On Adversarial Robustness of Large-scale Audio Visual Learning

Deeply Interleaved Two-Stream Encoder for Referring Video Segmentation

Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?

Multimodal Driver Referencing: A Comparison of Pointing to Objects Inside and Outside the Vehicle

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis

Multi-Modal Perception Attention Network with Self-Supervised Learning for Audio-Visual Speaker Tracking

Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues

TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment

HEU Emotion: A Large-scale Database for Multi-modal Emotion Recognition in the Wild

Interactive Speech and Noise Modeling for Speech Enhancement

Fusion Models for Improved Visual Captioning

VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs

Detecting Hate Speech in Multi-modal Memes

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

American Sign Language Recognition Using RF Sensing

KBGN: Knowledge-Bridge Graph Network for Adaptive Vision-Text Reasoning in Visual Dialogue

Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention

Audio- and Gaze-driven Facial Animation of Codec Avatars

Cross-Modal Weighting Network for RGB-D Salient Object Detection

Temporal aggregation of audio-visual modalities for emotion recognition

EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

Audio-Visual Kinship Verification

Cross-Modal Message Passing for Two-stream Fusion

Cross-Modal Attentional Context Learning for RGB-D Object Detection

Multimodal Polynomial Fusion for Detecting Driver Distraction

Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion

Multimodal Local-Global Ranking Fusion for Emotion Recognition

Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning

Audio-Visual Event Localization in Unconstrained Videos

Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression