多模态信息融合 - arXivDaily 专题

2410.20097 2024-10-29 cs.CV 57%

Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification

Yue Su, Hao Li, Maoguo Gong

专题命中音视频/视觉语言融合：visible-infrared(abstract)；分类 cs.CV

2410.17779 2024-10-24 cs.CV 57%

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Yonggang Wen

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2407.04955 2024-10-01 cs.CV 57%

Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations

Dingkang Yang, Mingcheng Li, Linhao Qu, Kun Yang, Peng Zhai, Song Wang, Lihua Zhang

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by TCSVT 2024

2409.15936 2024-09-25 cs.CY cs.CV cs.HC 57%

DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection

Jiaxin Ye, Junping Zhang, Hongming Shan

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2408.15566 2024-08-29 cs.CV 57%

TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center Learning

Jinglun Li, Xinyu Zhou, Kaixun Jiang, Lingyi Hong, Pinxue Guo, Zhaoyu Chen, Weifeng Ge, Wenqiang Zhang

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by ACMMM2024

2408.05711 2024-08-13 cs.CV 57%

Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval

Rukai Wei, Heng Cui, Yu Liu, Yufeng Hou, Yanzhao Xie, Ke Zhou

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments Accepted by ICME 2024

2407.21721 2024-08-01 cs.MM cs.AI 57%

Open-Vocabulary Audio-Visual Semantic Segmentation

Ruohao Guo, Liao Qu, Dantong Niu, Yanyu Qi, Wenzhen Yue, Ji Shi, Bowei Xing, Xianghua Ying

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.MM

Comments Accepted by ACM MM 2024 (Oral)

2407.19224 2024-07-31 cs.SD cs.MM eess.AS 57%

RAVSS: Robust Audio-Visual Speech Separation in Multi-Speaker Scenarios with Missing Visual Cues

Tianrui Pan, Jie Liu, Bohan Wang, Jie Tang, Gangshan Wu

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.MM

Comments Accepted by MM 2024

2407.06730 2024-07-10 cs.CV 57%

LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition

Teng Wang, Lingquan Meng, Lei Cheng, Changyin Sun

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

2407.02004 2024-07-08 cs.CV cs.AI cs.SD eess.AS 57%

SAVE: Segment Audio-Visual Easy way using Segment Anything Model

Khanh-Binh Nguyen, Chae Jung Park

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

2312.16862 2024-06-24 cs.CV cs.CL 57%

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

Zhengqing Yuan, Zhaoxu Li, Weiran Huang, Yanfang Ye, Lichao Sun

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments Accepted by ICML workshop 2024

2406.12355 2024-06-19 cs.CV 57%

LiCAF: LiDAR-Camera Asymmetric Fusion for Gait Recognition

Yunze Deng, Haijun Xiong, Bin Feng

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments Accepted by ICIP2024

2405.11205 2024-05-21 cs.CV 57%

Fuse & Calibrate: A bi-directional Vision-Language Guided Framework for Referring Image Segmentation

Yichen Yan, Xingjian He, Sihan Chen, Shichen Lu, Jing Liu

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments 12 pages, 4 figures ICIC2024

2405.07451 2024-05-14 cs.CV 57%

CLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answering

Yuanyuan Jiang, Jianqin Yin

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments Submitted to the Journal on February 6, 2024

2405.03351 2024-05-07 cs.CV 57%

Modality Prompts for Arbitrary Modality Salient Object Detection

Nianchang Huang, Yang Yang, Qiang Zhang, Jungong Han, Jin Huang

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments 13 pages, 7 Figures, 3 Tables

2403.04661 2024-04-23 cs.CV cs.LG cs.SD eess.AS 57%

Dynamic Cross Attention for Audio-Visual Person Verification

R. Gnana Praveen, Jahangir Alam

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments Accepted to FG2024

2404.13039 2024-04-22 cs.CV cs.CL 57%

LaPA: Latent Prompt Assist Model For Medical Visual Question Answering

Tiancheng Gu, Kaicheng Yang, Dongnan Liu, Weidong Cai

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

Comments 10 pages, 4 figures, Accepted by CVPRW2024

2403.11311 2024-03-26 cs.CL cs.MM 57%

Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding

Zichen Wu, Hsiu-Yuan Huang, Fanyi Qu, Yunfang Wu

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.MM

Comments LREC-COLING 2024, Long Paper

2403.06679 2024-03-12 cs.CV 57%

Answering Diverse Questions via Text Attached with Key Audio-Visual Clues

Qilang Ye, Zitong Yu, Xin Liu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2301.09209 2024-03-12 cs.CV cs.CL 57%

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

Razvan-George Pasca, Alexey Gavryushin, Muhammad Hamza, Yen-Ling Kuo, Kaichun Mo, Luc Van Gool, Otmar Hilliges, Xi Wang

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

2403.01700 2024-03-05 cs.SD cs.MM eess.AS 57%

Robust Wake Word Spotting With Frame-Level Cross-Modal Attention Based Audio-Visual Conformer

Haoxu Wang, Ming Cheng, Qiang Fu, Ming Li

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.MM

Comments Accepted by ICASSP 2024

2110.08021 2024-02-23 cs.LG cs.CL cs.MM 57%

StreaMulT: Streaming Multimodal Transformer for Heterogeneous and Arbitrary Long Sequential Data

Victor Pellegrain, Myriam Tami, Michel Batteux, Céline Hudelot

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.MM

Comments 11 pages, 6 figures, 3 tables

2305.18797 2024-02-14 cs.CV 57%

Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Space

Xiaogang Peng, Hao Wen, Yikai Luo, Xiao Zhou, Keyang Yu, Ping Yang, Zizhao Wu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.CV

Comments 11 pages, 12 figures, typos are fixed

2402.05457 2024-02-09 cs.CL cs.AI cs.MM cs.SD eess.AS 57%

It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition

Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng, Chao-Han Huck Yang

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.MM

Comments Accepted to ICLR 2024, 17 pages. This work will be open sourced under MIT license

2312.14471 2023-12-25 cs.CV 57%

Prototype-based Cross-Modal Object Tracking

Lei Liu, Chenglong Li, Futian Wang, Longfeng Shen, Jin Tang

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments In Peer Review

2312.12721 2023-12-21 cs.CV 57%

Cross-Modal Reasoning with Event Correlation for Video Question Answering

Chengxiang Yin, Zhengping Che, Kun Wu, Zhiyuan Xu, Qinru Qiu, Jian Tang

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)；分类 cs.CV

2305.12397 2023-12-11 cs.CV 57%

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

Yuanyuan Jiang, Jianqin Yin

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments Accepted to EMNLP 2023 Findings

2311.14275 2023-11-27 cs.CV cs.SD eess.AS 57%

Cooperative Dual Attention for Audio-Visual Speech Enhancement with Facial Cues

Feixiang Wang, Shuang Yang, Shiguang Shan, Xilin Chen

专题命中音视频/视觉语言融合：audio-visual fusion(abstract)；分类 cs.CV

Comments Accepted to BMVC 2023 15 pages, 2 figures

2207.12261 2023-11-23 cs.CL cs.LG cs.MM 57%

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

Jiang Li, Xiaoping Wang, Guoqing Lv, Zhigang Zeng

专题命中音视频/视觉语言融合：multimodal fusion(abstract)；分类 cs.MM

Comments Accepted by IEEE Transactions on Multimedia (TMM)

2305.13631 2023-10-24 cs.CL cs.CV cs.IR 57%

EDIS: Entity-Driven Image Search over Multimodal Web Content

Siqi Liu, Weixi Feng, Tsu-jui Fu, Wenhu Chen, William Yang Wang

专题命中音视频/视觉语言融合：information fusion(abstract)；分类 cs.CV

Comments EMNLP 2023 camera ready version

1. 音视频/视觉语言融合 495 篇

Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations

DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection

TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center Learning

Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval

Open-Vocabulary Audio-Visual Semantic Segmentation

RAVSS: Robust Audio-Visual Speech Separation in Multi-Speaker Scenarios with Missing Visual Cues

LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition

SAVE: Segment Audio-Visual Easy way using Segment Anything Model

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

LiCAF: LiDAR-Camera Asymmetric Fusion for Gait Recognition

Fuse & Calibrate: A bi-directional Vision-Language Guided Framework for Referring Image Segmentation

CLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answering

Modality Prompts for Arbitrary Modality Salient Object Detection

Dynamic Cross Attention for Audio-Visual Person Verification

LaPA: Latent Prompt Assist Model For Medical Visual Question Answering

Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding

Answering Diverse Questions via Text Attached with Key Audio-Visual Clues

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

Robust Wake Word Spotting With Frame-Level Cross-Modal Attention Based Audio-Visual Conformer

StreaMulT: Streaming Multimodal Transformer for Heterogeneous and Arbitrary Long Sequential Data

Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Space

It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition

Prototype-based Cross-Modal Object Tracking

Cross-Modal Reasoning with Event Correlation for Video Question Answering

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

Cooperative Dual Attention for Audio-Visual Speech Enhancement with Facial Cues

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

EDIS: Entity-Driven Image Search over Multimodal Web Content