多模态大模型

Reflective Teacher: Semi-Supervised Multimodal 3D Object Detection in Bird's-Eye-View via Uncertainty Measure

Saheli Hazra, Sudip Das, Rohit Choudhary, Arindam Das, Ganesh Sistu, Ciaran Eising, Ujjwal Bhattacharya

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Multimodal Remote Sensing Scene Classification Using VLMs and Dual-Cross Attention Networks

Jinjin Cai, Kexin Meng, Baijian Yang, Gang Shao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

GSIFN: A Graph-Structured and Interlaced-Masked Multimodal Transformer-based Fusion Network for Multimodal Sentiment Analysis

Yijie Jin

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Comments Withdraw for the error in the paper

Multimodal Fusion Learning with Dual Attention for Medical Imaging

Joy Dhar, Nayyar Zaidi, Maryam Haghighat, Puneet Goyal, Sudipta Roy, Azadeh Alavi, Vikas Kumar

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages

Journal ref IEEE/CVF Winter Conference on Applications of Computer Vision WACV 2025

Efficient Multi-modal Large Language Models via Visual Token Grouping

Minbin Huang, Runhui Huang, Han Shi, Yimeng Chen, Chuanyang Zheng, Xiangguo Sun, Xin Jiang, Zhenguo Li, Hong Cheng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality

Chenyang Lei, Liyi Chen, Jun Cen, Xiao Chen, Zhen Lei, Felix Heide, Qifeng Chen, Zhaoxiang Zhang

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Comments project page: https://mt-cly.github.io/SimCMF.github.io/. arXiv admin note: substantial text overlap with arXiv:2409.08083

Revisiting Misalignment in Multispectral Pedestrian Detection: A Language-Driven Approach for Cross-modal Alignment Fusion

Taeheon Kim, Sangyun Chung, Youngjoon Yu, Yong Man Ro

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

OracleSage: Towards Unified Visual-Linguistic Understanding of Oracle Bone Scripts through Cross-Modal Knowledge Fusion

Hanqi Jiang, Yi Pan, Junhao Chen, Zhengliang Liu, Yifan Zhou, Peng Shu, Yiwei Li, Huaqin Zhao, Stephen Mihm, Lewis C Howe, Tianming Liu

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Seeking the Sufficiency and Necessity Causal Features in Multimodal Representation Learning

Boyu Chen, Junjie Liu, Zhu Li, Mengyue Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization

Hongrui Jia, Chaoya Jiang, Haiyang Xu, Wei Ye, Mengfan Dong, Ming Yan, Ji Zhang, Fei Huang, Shikun Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus

Stephanie M. Lukin, Claire Bonial, Matthew Marge, Taylor Hudson, Cory J. Hayes, Kimberly A. Pollard, Anthony Baker, Ashley N. Foots, Ron Artstein, Felix Gervits, Mitchell Abrams, Cassidy Henry, Lucia Donatelli, Anton Leuski, Susan G. Hill, David Traum, Clare R. Voss

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL

Comments 14 pages, 7 figures

Journal ref 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) https://aclanthology.org/2024.lrec-main.1259/

QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou

Xinchen Luo, Jiangxia Cao, Tianyu Sun, Jinkai Yu, Rui Huang, Wei Yuan, Hezheng Lin, Yichen Zheng, Shiyao Wang, Qigen Hu, Changqing Qiu, Jiaqi Zhang, Xu Zhang, Zhiheng Yan, Jingming Zhang, Simin Zhang, Mingxing Wen, Zhaojie Liu, Kun Gai, Guorui Zhou

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Comments Work in progress

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model

Ting Liu, Liangtao Shi, Richang Hong, Yue Hu, Quanjun Yin, Linfeng Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 8 pages, 4figures

MDA: An Interpretable and Scalable Multi-Modal Fusion under Missing Modalities and Intrinsic Noise Conditions

Lin Fan, Yafei Ou, Cenyang Zheng, Pengyu Dai, Tamotsu Kamishima, Masayuki Ikebe, Kenji Suzuki, Xun Gong

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

A Multimodal Fusion Network For Student Emotion Recognition Based on Transformer and Tensor Product

Ao Xiang, Zongqing Qi, Han Wang, Qin Yang, Danqing Ma

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Models

Wei Wang, Zhaowei Li, Qi Xu, Linfeng Li, YiQing Cai, Botian Jiang, Hang Song, Xingcan Hu, Pengyu Wang, Li Xiao

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

CoMiX: Cross-Modal Fusion with Deformable Convolutions for HSI-X Semantic Segmentation

Xuming Zhang, Xingfa Gu, Qingjiu Tian, Lorenzo Bruzzone

专题命中 多模态训练与对齐 ：cross-modal(title)；multimodal(abstract)；分类 cs.CV

RNA-GPT: Multimodal Generative System for RNA Sequence Understanding

Yijia Xiao, Edward Sun, Yiqiao Jin, Wei Wang

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Comments Machine Learning for Structural Biology Workshop, NeurIPS 2024

AstroM$^3$: A self-supervised multimodal model for astronomy

Mariia Rizhko, Joshua S. Bloom

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Instruction Tuning with Hybrid State Space Models

Jianing Zhou, Han Li, Shuai Zhang, Ning Xie, Ruijie Wang, Xiaohan Nie, Sheng Liu, Lingyun Wang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Material Property Prediction with Element Attribute Knowledge Graphs and Multimodal Representation Learning

Chao Huang, Chunyan Chen, Ling Shi, Chen Chen

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Adapting Segment Anything Model to Multi-modal Salient Object Detection with Semantic Feature Fusion Guidance

Kunpeng Wang, Danying Lin, Chenglong Li, Zhengzheng Tu, Bin Luo

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments 10 pages, 9 figures

Multi-modal Iterative and Deep Fusion Frameworks for Enhanced Passive DOA Sensing via a Green Massive H2AD MIMO Receiver

Jiatong Bai, Minghao Chen, Wankai Tang, Yifan Li, Cunhua Pan, Yongpeng Wu, Feng Shu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Predicting Stroke through Retinal Graphs and Multimodal Self-supervised Learning

Yuqing Huang, Bastian Wittmann, Olga Demler, Bjoern Menze, Neda Davoudi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted as oral paper at ML-CDS workshop, MICCAI 2024

Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications

Daan Schouten, Giulia Nicoletti, Bas Dille, Catherine Chia, Pierpaolo Vendittelli, Megan Schuurmans, Geert Litjens, Nadieh Khalili

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 28 pages

Graph-Based Multi-Modal Sensor Fusion for Autonomous Driving

Depanshu Sani, Saket Anand

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments An extended abstract accepted at Young Researchers' Symposium, ICVGIP '24. This extended abstract contains the following: 1. Short summary of our work, SAGA-KF, accepted at ICPR'24. 2. A proposal that was awarded the Qualcomm Innovation Fellowship'24

Multimodal Super-Resolution: Discovering hidden physics and its application to fusion plasmas

Azarakhsh Jalalvand, SangKyeun Kim, Jaemin Seo, Qiming Hu, Max Curie, Peter Steiner, Andrew Oakleigh Nelson, Yong-Su Na, Egemen Kolemen

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Multi-modal biometric authentication: Leveraging shared layer architectures for enhanced security

Vatchala S, Yogesh C, Yeshwanth Govindarajan, Krithik Raja M, Vishal Pramav Amirtha Ganesan, Aashish Vinod A, Dharun Ramesh

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based Inference

Marvin Schmitt, Leona Odole, Stefan T. Radev, Paul-Christian Bürkner

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Graph Neural Network for Recommendation with Dynamic De-redundancy and Modality-Guided Feature De-noisy

Feng Mo, Lin Xiao, Qiya Song, Xieping Gao, Eryao Liang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.MM