多模态大模型 - arXivDaily 专题

2409.15745 2024-09-25 eess.IV cs.CV 79%

ManiNeg: Manifestation-guided Multimodal Pretraining for Mammography Classification

Xujun Li, Xin Wei, Jing Jiang, Danxiang Chen, Wei Zhang, Jinpeng Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2409.15744 2024-09-25 eess.IV cs.CV 79%

ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features

Xin Wei, Yaling Tao, Changde Du, Gangming Zhao, Yizhou Yu, Jinpeng Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2403.09212 2024-09-24 cs.CV 79%

PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest

Jiajun Deng, Sha Zhang, Feras Dayoub, Wanli Ouyang, Yanyong Zhang, Ian Reid

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments https://djiajunustc.github.io/projects/poifusion

2409.11983 2024-09-19 cs.CV 79%

Intraoperative Registration by Cross-Modal Inverse Neural Rendering

Maximilian Fehrentz, Mohammad Farid Azampour, Reuben Dorent, Hassan Rasheed, Colin Galvin, Alexandra Golby, William M. Wells, Sarah Frisken, Nassir Navab, Nazim Haouchine

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at MICCAI 2024

2403.11577 2024-09-19 cs.CV cs.RO 79%

3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration

Quentin Herau, Moussab Bennehar, Arthur Moreau, Nathan Piasco, Luis Roldao, Dzmitry Tsishkou, Cyrille Migniot, Pascal Vasseur, Cédric Demonceaux

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at IROS 2024 (Oral presentation). Project page: https://qherau.github.io/3DGS-Calib/

2409.08582 2024-09-16 cs.CV 79%

ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning

Pei Deng, Wenqian Zhou, Hanlin Wu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 5 pages, 2 figures

2409.08543 2024-09-16 cs.IR cs.AI 79%

ATFLRec: A Multimodal Recommender System with Audio-Text Fusion and Low-Rank Adaptation via Instruction-Tuned Large Language Model

Zezheng Qin

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

2409.07855 2024-09-13 cs.CE cs.MM 79%

MSMF: Multi-Scale Multi-Modal Fusion for Enhanced Stock Market Prediction

Jiahao Qin

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.MM

Comments 15 pages, 1 figures, 7 tables

2409.05015 2024-09-11 cs.HC cs.SD eess.AS 79%

Improving Multimodal Emotion Recognition by Leveraging Acoustic Adaptation and Visual Alignment

Zhixian Zhao, Haifeng Chen, Xi Li, Dongmei Jiang, Lei Xie

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 eess.AS

2409.05916 2024-09-11 q-bio.QM cs.AI cs.LG q-bio.BM 79%

Unlocking Potential Binders: Multimodal Pretraining DEL-Fusion for Denoising DNA-Encoded Libraries

Chunbin Gu, Mutian He, Hanqun Cao, Guangyong Chen, Chang-yu Hsieh, Pheng Ann Heng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

2404.11119 2024-09-10 cs.IR cs.MM 79%

DREAM: A Dual Representation Learning Model for Multimodal Recommendation

Kangning Zhang, Yingjie Qin, Jiarui Jin, Yifan Liu, Ruilong Su, Weinan Zhang, Yong Yu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

Comments 10 pages, 11 figures

2409.04563 2024-09-10 cs.CV 79%

Influence of Early through Late Fusion on Pancreas Segmentation from Imperfectly Registered Multimodal MRI

Lucas W. Remedios, Han Liu, Samuel W. Remedios, Lianrui Zuo, Adam M. Saunders, Shunxing Bao, Yuankai Huo, Alvin C. Powers, John Virostko, Bennett A. Landman

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 13.5 pages of manuscript content

2403.04650 2024-09-10 cs.LG cs.AI 79%

Lightweight Cross-Modal Representation Learning

Bilal Faye, Hanane Azzag, Mustapha Lebbah, Djamel Bouchaffra

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.AI

Journal ref ESANN 2024

2409.01151 2024-09-04 cs.CV cs.LG 79%

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Yueqian Wang, Jianxin Liang, Yuxuan Wang, Huishuai Zhang, Dongyan Zhao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2312.16778 2024-09-04 cs.CL 79%

Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition

Yuntao Shou, Tao Meng, Wei Ai, Nan Yin, Keqin Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments 14 pages, 6 figures

2311.05348 2024-08-29 cs.CV 79%

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Fanyi Wang, Yanchun Xie, Yi-Jie Huang, Yaqian Li

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2408.13830 2024-08-28 cs.CV 79%

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Yuhong Jiao, Jiaqing Miao, Jinnan Gong, Hui He, Ping Liang, Cheng Luo, Ying Tan

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.14387 2024-08-27 cs.LG cs.AI 79%

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning

Sakhinana Sagar Srinivas, Chidaksh Ravuru, Geethan Sannidhi, Venkataramana Runkana

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.AI

Comments Paper published at the Deployable AI (DAI) workshop at AAAI-2024

2408.13622 2024-08-27 cs.LG cs.AI 79%

Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings

Sagar Srinivas Sakhinana, Geethan Sannidhi, Chidaksh Ravuru, Venkataramana Runkana

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

Comments Published at the ICLR 2024 Workshop on Practical ML for Low Resource Settings(PML4LRS)

2308.10421 2024-08-26 cs.CV 79%

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving

Jian Zou, Tianyu Huang, Guanglei Yang, Zhenhua Guo, Tao Luo, Chun-Mei Feng, Wangmeng Zuo

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Code available at https://github.com/hollow-503/UniM2AE

2407.19156 2024-08-20 cs.CV 79%

Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble

Juhan Cha, Minseok Joo, Jihwan Park, Sanghyeok Lee, Injae Kim, Hyunwoo J. Kim

专题命中多模态训练与对齐：multimodal(title)；multi-modal(abstract)；分类 cs.CV

2309.05919 2024-08-20 eess.IV cs.CV 79%

Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

Ling Huang, Su Ruan, Pierre Decazes, Thierry Denoeux

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.09064 2024-08-20 cs.CV cs.LG 79%

MoRA: LoRA Guided Multi-Modal Disease Diagnosis with Missing Modality

Zhiyi Shi, Junsik Kim, Wanhua Li, Yicong Li, Hanspeter Pfister

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

2408.08564 2024-08-19 cs.IR cs.CL 79%

Collaborative Cross-modal Fusion with Large Language Model for Recommendation

Zhongzhou Liu, Hao Zhang, Kuicai Dong, Yuan Fang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CL

Comments 10 pages, 4 figures, accepted by CIKM 2024

2308.07222 2024-08-15 cs.IR cs.AI 79%

MM-GEF: Multi-modal representation meet collaborative filtering

Hao Wu, Alejandro Ariza-Casabona, Bartłomiej Twardowski, Tri Kurniawan Wijaya

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2408.05751 2024-08-13 cs.IR cs.CV 79%

Advancing Re-Ranking with Multimodal Fusion and Target-Oriented Auxiliary Tasks in E-Commerce Search

Enqiang Xu, Xinhui Li, Zhigong Zhou, Jiahao Ji, Jinyuan Zhao, Dadong Miao, Songlin Wang, Lin Liu, Sulong Xu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.04547 2024-08-09 cs.MM 79%

Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation

Haoxiang Shi, Ziqi Liang, Jun Yu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.MM

Comments Accepted by INTERSPEECH 2024

2408.04170 2024-08-09 cs.CV 79%

M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction

Hui Luo, Jiashuang Huang, Hengrong Ju, Tianyi Zhou, Weiping Ding

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.02503 2024-08-06 cs.CL 79%

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model

Zhaowei Li, Wei Wang, YiQing Cai, Xu Qi, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL

2408.00779 2024-08-05 cs.LG cs.AI cs.ET cs.IT math.IT q-bio.BM 79%

Learning Structurally Stabilized Representations for Multi-modal Lossless DNA Storage

Ben Cao, Tiantian He, Xue Li, Bin Wang, Xiaohu Wu, Qiang Zhang, Yew-Soon Ong

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

1. 多模态训练与对齐 6887 篇

ManiNeg: Manifestation-guided Multimodal Pretraining for Mammography Classification

ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features

PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest

Intraoperative Registration by Cross-Modal Inverse Neural Rendering

3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration

ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning

ATFLRec: A Multimodal Recommender System with Audio-Text Fusion and Low-Rank Adaptation via Instruction-Tuned Large Language Model

MSMF: Multi-Scale Multi-Modal Fusion for Enhanced Stock Market Prediction

Improving Multimodal Emotion Recognition by Leveraging Acoustic Adaptation and Visual Alignment

Unlocking Potential Binders: Multimodal Pretraining DEL-Fusion for Denoising DNA-Encoded Libraries

DREAM: A Dual Representation Learning Model for Multimodal Recommendation

Influence of Early through Late Fusion on Pancreas Segmentation from Imperfectly Registered Multimodal MRI

Lightweight Cross-Modal Representation Learning

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning

Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving

Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble

Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

MoRA: LoRA Guided Multi-Modal Disease Diagnosis with Missing Modality

Collaborative Cross-modal Fusion with Large Language Model for Recommendation

MM-GEF: Multi-modal representation meet collaborative filtering

Advancing Re-Ranking with Multimodal Fusion and Target-Oriented Auxiliary Tasks in E-Commerce Search

Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation

M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model

Learning Structurally Stabilized Representations for Multi-modal Lossless DNA Storage