多模态大模型

SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction

Ying Chen, Jiajing Xie, Yuxiang Lin, Yuhang Song, Wenxian Yang, Rongshan Yu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics

Jordan J. Bird

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

freePruner: A Training-free Approach for Large Multimodal Model Acceleration

Bingxin Xu, Yuzhang Shang, Yunhao Ge, Qian Lou, Yan Yan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Unsupervised Homography Estimation on Multimodal Image Pair via Alternating Optimization

Sanghyeob Song, Jaihyun Lew, Hyemi Jang, Sungroh Yoon

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments This paper is accepted to the Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)

Edge-Enhanced Dilated Residual Attention Network for Multimodal Medical Image Fusion

Meng Zhou, Yuxuan Zhang, Xiaolan Xu, Jiayi Wang, Farzad Khalvati

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments An extended version of the paper accepted at IEEE BIBM 2024

CMATH: Cross-Modality Augmented Transformer with Hierarchical Variational Distillation for Multimodal Emotion Recognition in Conversation

Xiaofei Zhu, Jiawei Cheng, Zhou Yang, Zhuo Chen, Qingyang Wang, Jianfeng Yao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

VMID: A Multimodal Fusion LLM Framework for Detecting and Identifying Misinformation of Short Videos

Weihao Zhong, Yinhao Xiao, Minghui Xu, Xiuzhen Cheng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2211.10973 by other authors

Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning

Mingcheng Li, Dingkang Yang, Yang Liu, Shunli Wang, Jiawei Chen, Shuaibing Wang, Jinjie Wei, Yue Jiang, Qingyao Xu, Xiaolu Hou, Mingyang Sun, Ziyun Qian, Dongliang Kou, Lihua Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by NeurIPS 2024

EEG-based Multimodal Representation Learning for Emotion Recognition

Kang Yin, Hye-Bin Shin, Dan Li, Seong-Whan Lee

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, Alexander Hauptmann

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted at NeurIPS 2024. 49 pages, 13 figures, Project: https://github.com/ZebangCheng/Emotion-LLaMA, Demo: https://huggingface.co/spaces/ZebangCheng/Emotion-LLaMA

Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Grounding

Jinlong He, Pengfei Li, Gang Liu, Shenjun Zhong

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning

Taowen Wang, Yiyang Liu, James Chenhao Liang, junhan zhao, Yiming Cui, Yuning Mao, Shaoliang Nie, Jiahao Liu, Fuli Feng, Zenglin Xu, Cheng Han, Lifu Huang, Qifan Wang, Dongfang Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance

Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip H. S. Torr, Adel Bibi, Samuel Albanie, Matthias Bethge

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Short version accepted at DPFM, ICLR'24; Full paper at NeurIPS'24

AMPLE: Emotion-Aware Multimodal Fusion Prompt Learning for Fake News Detection

Xiaoman Xu, Xiangrun Li, Taihang Wang, Ye Jiang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate

Qidong Huang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin, Weiming Zhang, Nenghai Yu

专题命中 多模态训练与对齐 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://github.com/shikiw/Modality-Integration-Rate

ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy

Hong Li, Zhiquan Tan, Xingyu Li, Weiran Huang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Learning Content-Aware Multi-Modal Joint Input Pruning via Bird's-Eye-View Representation

Yuxin Li, Yiheng Li, Xulei Yang, Mengying Yu, Zihang Huang, Xiaojun Wu, Chai Kiat Yeo

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Fusion Strategies for Mapping Biophysical Landscape Features

Lucia Gordon, Nico Lang, Catherine Ressijac, Andrew Davies

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 4 figures, ECCV 2024 Workshop in CV for Ecology

Multimodal 3D Fusion and In-Situ Learning for Spatially Aware AI

Chengyuan Xu, Radha Kumaran, Noah Stier, Kangyou Yu, Tobias Höllerer

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 6 figures, accepted to IEEE ISMAR 2024

Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification

Salma Hassan, Hamad Al Hammadi, Ibrahim Mohammed, Muhammad Haris Khan

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs

Mustafa Shukor, Matthieu Cord

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2024. Code: https://github.com/mshukor/ima-lmms. Project page: https://ima-lmms.github.io/

Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats

Kuanrong Liu, Siyuan Liang, Jiawei Liang, Pengwen Dai, Xiaochun Cao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Alternative Telescopic Displacement: An Efficient Multimodal Alignment Method

Jiahao Qin, Yitao Xu, Zong Lu, Xiaojun Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 1 figures

Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models

Venkat Karramreddy, Liam Mitchell

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 10 figures

A Novel Adaptive Fine-Tuning Algorithm for Multimodal Models: Self-Optimizing Classification and Selection of High-Quality Datasets in Remote Sensing

Yi Ren, Tianyi Zhang, Zhixiong Han, Weibin Li, Zhiyang Wang, Wenbo Ji, Chenhao Qin, Chenbin Liang, Licheng Jiao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task

Yanjie Li, Jingyi Liu, Weijun Li, Lina Yu, Min Wu, Wenqiang Li, Meilan Hao, Su Wei, Yusong Deng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments The Information Fusion has accepted this paper

Learning Noise-Robust Joint Representation for Multimodal Emotion Recognition under Incomplete Data Scenarios

Qi Fan, Haolin Zuo, Rui Liu, Zheng Lian, Guanglai Gao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by MRAC@ACM Multimedia 2024

Multi-modal Relation Distillation for Unified 3D Representation Learning

Huiqun Wang, Yiping Bao, Panwang Pan, Zeming Li, Xiao Liu, Ruijie Yang, Di Huang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV2024

CAST: Cross-modal Alignment Similarity Test for Vision Language Models

Gautier Dagan, Olga Loginova, Anil Batra

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion

Yingxuan Li, Ryota Hinami, Kiyoharu Aizawa, Yusuke Matsui

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM Multimedia 2024. Project page: https://liyingxuan1012.github.io/zeroshot-speaker-prediction ; Github repo: https://github.com/liyingxuan1012/zeroshot-speaker-prediction