多模态大模型 - arXivDaily 专题

2304.03916 2023-06-01 cs.LG cs.AI 79%

Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning

Yu Yang, Besmira Nushi, Hamid Palangi, Baharan Mirzasoleiman

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2305.11439 2023-05-22 cs.CV 79%

Few-Shot Learning with Visual Distribution Calibration and Cross-Modal Distribution Alignment

Runqi Wang, Hao Zheng, Xiaoyue Duan, Jianzhuang Liu, Yuning Lu, Tian Wang, Songcen Xu, Baochang Zhang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

2210.14616 2023-05-16 cs.CR cs.AI 79%

A Late Multi-Modal Fusion Model for Detecting Hybrid Spam E-mail

Zhibo Zhang, Ernesto Damiani, Hussam Al Hamadi, Chan Yeob Yeun, Fatma Taher

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

Comments The content of this paper needs to be updated

Journal ref Index in journal International Journal of Computer Theory and Engineering (IJCTE), 2023

2212.01923 2023-05-11 cs.DB cs.AI cs.LG 79%

Query-Driven Knowledge Base Completion using Multimodal Path Fusion over Multimodal Knowledge Graph

Yang Peng, Daisy Zhe Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

2206.06488 2023-05-11 cs.CV cs.LG 79%

Multimodal Learning with Transformers: A Survey

Peng Xu, Xiatian Zhu, David A. Clifton

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments This paper is accepted by IEEE TPAMI

2208.10240 2023-05-10 cs.CL 79%

A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction

Weimin Lyu, Xinyu Dong, Rachel Wong, Songzhu Zheng, Kayley Abell-Hart, Fusheng Wang, Chao Chen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments AMIA Annual Symposium Proceedings 2022

Journal ref AMIA Annu Symp Proc 2022

2305.03810 2023-05-09 cs.HC cs.CV 79%

Distilled Mid-Fusion Transformer Networks for Multi-Modal Human Activity Recognition

Jingcheng Li, Lina Yao, Binghao Li, Claude Sammut

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 13 pages, 6 figures

2211.07098 2023-05-09 cs.AI cs.DB cs.LG 79%

Knowledge Base Completion using Web-Based Question Answering and Multimodal Fusion

Yang Peng, Daisy Zhe Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

2209.06954 2023-05-09 cs.CV 79%

Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering

Jingjing Jiang, Ziyi Liu, Nanning Zheng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 20 pages, 6 figures, 13 tables

2305.00201 2023-05-02 cs.CV 79%

Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViT

Zhenxiang Xiao, Yuzhong Chen, Lu Zhang, Junjie Yao, Zihao Wu, Xiaowei Yu, Yi Pan, Lin Zhao, Chong Ma, Xinyu Liu, Wei Liu, Xiang Li, Yixuan Yuan, Dinggang Shen, Dajiang Zhu, Tianming Liu, Xi Jiang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2304.05351 2023-05-01 cs.CL cs.LG q-fin.ST 79%

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

Qianqian Xie, Weiguang Han, Yanzhao Lai, Min Peng, Jimin Huang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments 13 pages

2304.11979 2023-04-25 cs.IR cs.MM 79%

Attention-guided Multi-step Fusion: A Hierarchical Fusion Network for Multimodal Recommendation

Yan Zhou, Jie Guo, Hao Sun, Bin Song, Fei Richard Yu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

2304.11381 2023-04-25 cs.CV 79%

Incomplete Multimodal Learning for Remote Sensing Data Fusion

Yuxing Chen, Maofan Zhao, Lorenzo Bruzzone

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2304.09694 2023-04-20 cs.CV 79%

CrossFusion: Interleaving Cross-modal Complementation for Noise-resistant 3D Object Detection

Yang Yang, Weijie Ma, Hao Chen, Linlin Ou, Xinyi Yu

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

2304.07675 2023-04-18 cs.CV 79%

Multimodal Representation Learning of Cardiovascular Magnetic Resonance Imaging

Jielin Qiu, Peide Huang, Makiya Nakashima, Jaehyun Lee, Jiacheng Zhu, Wilson Tang, Pohao Chen, Christopher Nguyen, Byung-Hak Kim, Debbie Kwon, Douglas Weber, Ding Zhao, David Chen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 24 pages

2304.07147 2023-04-17 eess.IV cs.CV cs.LG 79%

Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection

Ashay Patel, Petru-Danial Tudiosu, Walter H. L. Pinaya, Gary Cook, Vicky Goh, Sebastien Ourselin, M. Jorge Cardoso

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:006

Journal ref Machine.Learning.for.Biomedical.Imaging. 2 (2023)

2303.09164 2023-04-17 cs.CV 79%

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

Jia Li, Yin Chen, Xuesong Zhang, Jiantao Nie, Ziqiang Li, Yangchen Yu, Yan Zhang, Richang Hong, Meng Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments Solutions of HFUT-CVers Team at the 5th ABAW Competition (CVPR 2023 workshop)

2304.06485 2023-04-14 eess.SP cs.AI cs.LG 79%

CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities

Konstantinos Kontras, Christos Chatzichristos, Huy Phan, Johan Suykens, Maarten De Vos

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments 10 pages, 4 figures, 2 tables, journal

2304.02853 2023-04-07 cs.CV 79%

Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce

Yang Jin, Yongzhi Li, Zehuan Yuan, Yadong Mu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 10 figures, accepted by CVPR 2023

2304.01563 2023-04-05 cs.CL 79%

Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment

Qian Li, Shu Guo, Yangyifei Luo, Cheng Ji, Lihong Wang, Jiawei Sheng, Jianxin Li

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL

2303.13810 2023-03-27 cs.CV eess.IV 79%

Evidence-aware multi-modal data fusion and its application to total knee replacement prediction

Xinwen Liu, Jing Wang, S. Kevin Zhou, Craig Engstrom, Shekhar S. Chandra

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2303.13802 2023-03-27 cs.CV 79%

Decoupled Multimodal Distilling for Emotion Recognition

Yong Li, Yuanzhi Wang, Zhen Cui

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments To appear at CVPR 2023, selected as a hightlight, 10% of accepted papers, 2.5% of submissions

2303.13101 2023-03-24 cs.CV 79%

MMFormer: Multimodal Transformer Using Multiscale Self-Attention for Remote Sensing Image Classification

Bo Zhang, Zuheng Ming, Wei Feng, Yaqian Liu, Liang He, Kaixing Zhao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2302.01392 2023-03-24 cs.CV 79%

Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion

Yiming Sun, Bing Cao, Pengfei Zhu, Qinghua Hu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2303.10816 2023-03-21 cs.AI cs.IR 79%

IMF: Interactive Multimodal Fusion Model for Link Prediction

Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, Chunxiao Xing

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments 9 pages, 7 figures, 4 tables, WWW'2023

2303.07126 2023-03-14 eess.IV cs.CV 79%

Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging

Zdravko Marinov, Simon Reiß, David Kersting, Jens Kleesiek, Rainer Stiefelhagen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 8 pages; 8 figures; 5 tables

2303.05302 2023-03-10 eess.IV cs.CV 79%

M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities

Hong Liu, Dong Wei, Donghuan Lu, Jinghan Sun, Liansheng Wang, Yefeng Zheng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Journal ref AAAI 2023

2106.12735 2023-03-08 cs.CV 79%

Multi-Modal 3D Object Detection in Autonomous Driving: a Survey

Yingjie Wang, Qiuyu Mao, Hanqi Zhu, Jiajun Deng, Yu Zhang, Jianmin Ji, Houqiang Li, Yanyong Zhang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by International Journal of Computer Vision (IJCV)

2211.11393 2023-03-03 cs.CV 79%

TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis

Yilan Zhang, Fengying Xie, Jianqi Chen

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 6 figures

2208.03028 2023-03-03 eess.IV cs.CV 79%

Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume

Wei Dai, Ziyao Zhang, Lixia Tian, Shengyuan Yu, Shuhui Wang, Zhao Dong, Hairong Zheng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

1. 多模态训练与对齐 6887 篇

Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning

Few-Shot Learning with Visual Distribution Calibration and Cross-Modal Distribution Alignment

A Late Multi-Modal Fusion Model for Detecting Hybrid Spam E-mail

Query-Driven Knowledge Base Completion using Multimodal Path Fusion over Multimodal Knowledge Graph

Multimodal Learning with Transformers: A Survey

A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction

Distilled Mid-Fusion Transformer Networks for Multi-Modal Human Activity Recognition

Knowledge Base Completion using Web-Based Question Answering and Multimodal Fusion

Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering

Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViT

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

Attention-guided Multi-step Fusion: A Hierarchical Fusion Network for Multimodal Recommendation

Incomplete Multimodal Learning for Remote Sensing Data Fusion

CrossFusion: Interleaving Cross-modal Complementation for Noise-resistant 3D Object Detection

Multimodal Representation Learning of Cardiovascular Magnetic Resonance Imaging

Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities

Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce

Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment

Evidence-aware multi-modal data fusion and its application to total knee replacement prediction

Decoupled Multimodal Distilling for Emotion Recognition

MMFormer: Multimodal Transformer Using Multiscale Self-Attention for Remote Sensing Image Classification

Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion

IMF: Interactive Multimodal Fusion Model for Link Prediction

Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging

M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities

Multi-Modal 3D Object Detection in Autonomous Driving: a Survey

TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis

Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume