多模态大模型

A Visual Question Answering Method for SAR Ship: Breaking the Requirement for Multimodal Dataset Construction and Model Fine-Tuning

Fei Wang, Chengcheng Chen, Hongyu Chen, Yugang Chang, Weiming Zeng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration

Kaiang Wen, Bin Xie, Bin Duan, Yan Yan

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

H2G2-Net: A Hierarchical Heterogeneous Graph Generative Network Framework for Discovery of Multi-Modal Physiological Responses

Haidong Gu, Nathan Gaw, Yinan Wang, Chancellor Johnstone, Christine Beauchene, Sophia Yuditskaya, Hrishikesh Rao, Chun-An Chou

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Comments Paper accepted in Human-Centric Representation Learning workshop at AAAI 2024 (https://hcrl-workshop.github.io/2024/)

PathoGen-X: A Cross-Modal Genomic Feature Trans-Align Network for Enhanced Survival Prediction from Histopathology Images

Akhila Krishna, Nikhil Cherian Kurian, Abhijeet Patil, Amruta Parulekar, Amit Sethi

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Multimodal Fusion on Low-quality Data: A Comprehensive Survey

Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Di Hu, Changqing Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Feel free to comment on our manuscript: qingyangzhang@tju$.$edu$.$cn

JEMA: A Joint Embedding Framework for Scalable Co-Learning with Multimodal Alignment

Joao Sousa, Roya Darabi, Armando Sousa, Frank Brueckner, Luís Paulo Reis, Ana Reis

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 26 pages, 14 figures

Enhanced Survival Prediction in Head and Neck Cancer Using Convolutional Block Attention and Multimodal Data Fusion

Aiman Farooq, Utkarsh Sharma, Deepak Mishra

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments Accepted to [ACCV 2024 Workshop]

DeepLight: Reconstructing High-Resolution Observations of Nighttime Light With Multi-Modal Remote Sensing Data

Lixian Zhang, Runmin Dong, Shuai Yuan, Jinxiao Zhang, Mengxuan Chen, Juepeng Zheng, Haohuan Fu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments This paper has been accepted in IJCAI 2024

BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignment

Mehdi Hosseinzadeh, Ian Reid

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted for presentation at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024. Project page: https://m80hz.github.io/bevpose/

Evaluating Cost-Accuracy Trade-offs in Multimodal Search Relevance Judgements

Silvia Terragni, Hoang Cuong, Joachim Daiber, Pallavi Gudipati, Pablo N. Mendes

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Journal ref CIKM MMSR 2024

Multi-modal Image and Radio Frequency Fusion for Optimizing Vehicle Positioning

Ouwen Huan, Tao Luo, Mingzhe Chen

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Cross-Sim-NGF: FFT-Based Global Rigid Multimodal Alignment of Image Volumes using Normalized Gradient Fields

Johan Öfverstedt, Joakim Lindblad, Nataša Sladoje

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 5 pages, 3 figures, 3 tables. This work has been submitted to the IEEE for possible publication

MEM: Multi-Modal Elevation Mapping for Robotics and Learning

Gian Erni, Jonas Frey, Takahiro Miki, Matias Mattamala, Marco Hutter

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accapted for IROS2023. This work has been submitted to the IEEE for possible publication

MolMix: A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning

Andrei Manolache, Dragos Tantaru, Mathias Niepert

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Machine Learning for Structural Biology Workshop, NeurIPS 2024 v2: Added optimizer references

Gradient-based Jailbreak Images for Multimodal Fusion Models

Javier Rando, Hannah Korevaar, Erik Brinkman, Ivan Evtimov, Florian Tramèr

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Reliable Object Tracking by Multimodal Hybrid Feature Extraction and Transformer-Based Fusion

Hongze Sun, Rui Liu, Wuque Cai, Jun Wang, Yue Wang, Huajin Tang, Yan Cui, Dezhong Yao, Daqing Guo

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 16 pages, 7 figures, 9 tabes; This work has been submitted for possible publication

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

Wenfang Yao, Kejing Yin, William K. Cheung, Jia Liu, Jing Qin

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by AAAI-24

PlaneSAM: Multimodal Plane Instance Segmentation Using the Segment Anything Model

Zhongchen Deng, Zhechen Yang, Chi Chen, Cheng Zeng, Yan Meng, Bisheng Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments submitted to Information Fusion

MCSFF: Multi-modal Consistency and Specificity Fusion Framework for Entity Alignment

Wei Ai, Wen Deng, Hongyi Chen, Jiayi Du, Tao Meng, Yuntao Shou

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Comments 6 pages, 1 figures

MultiMAE-DER: Multimodal Masked Autoencoder for Dynamic Emotion Recognition

Peihao Xiang, Chaohao Lin, Kaida Wu, Ou Bai

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Camera-ready Version, Accepted by ICPRS 2024

MMCFND: Multimodal Multilingual Caption-aware Fake News Detection for Low-resource Indic Languages

Shubhi Bansal, Nishit Sushil Singh, Shahid Shafi Dar, Nagendra Kumar

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Multi-modal Fusion based Q-distribution Prediction for Controlled Nuclear Fusion

Shiao Wang, Yifeng Wang, Qingchuan Ma, Xiao Wang, Ning Yan, Qingquan Yang, Guosheng Xu, Jin Tang

专题命中 多模态训练与对齐 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

MMLF: Multi-modal Multi-class Late Fusion for Object Detection with Uncertainty Estimation

Qihang Yang, Yang Zhao, Hong Cheng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow

Cyrile Delestre, Yoann Sola

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Representation Learning using Adaptive Graph Construction

Weichen Huang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

PRFusion: Toward Effective and Robust Multi-Modal Place Recognition with Image and Point Cloud Fusion

Sijie Wang, Qiyu Kang, Rui She, Kai Zhao, Yang Song, Wee Peng Tay

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments accepted by IEEE TITS 2024

Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning

Jianxiong Li, Zhihao Wang, Jinliang Zheng, Xiaoai Zhou, Guanming Wang, Guanglu Song, Yu Liu, Jingjing Liu, Ya-Qin Zhang, Junzhi Yu, Xianyuan Zhan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments preprint

Multimodal Alignment of Histopathological Images Using Cell Segmentation and Point Set Matching for Integrative Cancer Analysis

Jun Jiang, Raymond Moore, Brenna Novotny, Leo Liu, Zachary Fogarty, Ray Guo, Markovic Svetomir, Chen Wang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments initial version

Survival Prediction in Lung Cancer through Multi-Modal Representation Learning

Aiman Farooq, Deepak Mishra, Santanu Chaudhury

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted in WACV 2025

Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning

Wenwen Zhuang, Xin Huang, Xiantao Zhang, Jin Zeng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL