多模态大模型 - arXivDaily 专题

2408.14270 2024-08-27 eess.IV cs.CV 79%

Reliable Multi-modal Medical Image-to-image Translation Independent of Pixel-wise Aligned Data

Langrui Zhou, Guang Li

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.CV

Comments This paper has been accepted as a research article by Medical Physics

2408.13290 2024-08-27 eess.IV cs.CV 79%

Multi-modal Intermediate Feature Interaction AutoEncoder for Overall Survival Prediction of Esophageal Squamous Cell Cancer

Chengyu Wu, Yatao Zhang, Yaqi Wang, Qifeng Wang, Shuai Wang

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by ISBI 2024

2408.11761 2024-08-22 cs.RO cs.AI 79%

D-RMGPT: Robot-assisted collaborative tasks driven by large multimodal models

M. Forlini, M. Babcinschi, G. Palmieri, P. Neto

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

2408.11733 2024-08-22 cs.CV 79%

Enhancing Cross-Modal Medical Image Segmentation through Compositionality

Aniek Eijpe, Valentina Corbetta, Kalina Chupetlovska, Regina Beets-Tan, Wilson Silva

专题命中其他多模态：cross-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 3 figures, 2 tables. Accepted at Deep Generative Models workshop @ MICCAI 2024 (DGM4MICCAI). This is the submitted manuscript with added link to github repo, funding acknowledgements and authors' names and affiliations. No further post submission improvements or corrections were integrated. Final version not published yet

2408.00305 2024-08-02 cs.MM cs.IR 79%

Leveraging Weak Cross-Modal Guidance for Coherence Modelling via Iterative Learning

Yi Bin, Junrong Liao, Yujuan Ding, Haoxuan Li, Yang Yang, See-Kiong Ng, Heng Tao Shen

专题命中其他多模态：cross-modal(title,abstract)；分类 cs.MM

Comments Accepted by ACM Multimedia 2024

2311.11273 2024-07-31 cs.CV 79%

Chain of Visual Perception: Harnessing Multimodal Large Language Models for Zero-shot Camouflaged Object Detection

Lv Tang, Peng-Tao Jiang, Zhihao Shen, Hao Zhang, Jinwei Chen, Bo Li

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by ACMMM2024

2407.18251 2024-07-26 cs.CV cs.CR cs.LG 79%

Sparse vs Contiguous Adversarial Pixel Perturbations in Multimodal Models: An Empirical Analysis

Cristian-Alexandru Botocan, Raphael Meier, Ljiljana Dolamic

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

2407.16243 2024-07-24 cs.CV 79%

Chameleon: Images Are What You Need For Multimodal Learning Robust To Missing Modalities

Muhammad Irzam Liaqat, Shah Nawaz, Muhammad Zaigham Zaheer, Muhammad Saad Saeed, Hassan Sajjad, Tom De Schepper, Karthik Nandakumar, Muhammad Haris Khan Markus Schedl

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

2309.05300 2024-07-22 cs.CV 79%

Decoupling Common and Unique Representations for Multimodal Self-supervised Learning

Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Chenying Liu, Zhitong Xiong, Xiao Xiang Zhu

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to ECCV 2024. 27 pages, 8 figures

2404.02254 2024-07-18 stat.ML cs.AI cs.LG 79%

On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning

Ari Karchmer

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

Comments Appeared in ICML 2024. Camera-ready version

2407.10874 2024-07-16 cs.HC cs.CV cs.LG 79%

Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals

Keshav Bimbraw, Jing Liu, Ye Wang, Toshiaki Koike-Akino

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

Comments 5 pages, 4 figures

2407.10164 2024-07-16 cs.CV 79%

LabelDistill: Label-guided Cross-modal Knowledge Distillation for Camera-based 3D Object Detection

Sanmin Kim, Youngseok Kim, Sihwan Hwang, Hyeonjun Jeong, Dongsuk Kum

专题命中其他多模态：cross-modal(title,abstract)；分类 cs.CV

Comments ECCV 2024

2304.05645 2024-07-16 cs.CV 79%

WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language

Zhenxiang Lin, Xidong Peng, Peishan Cong, Ge Zheng, Yujin Sun, Yuenan Hou, Xinge Zhu, Sibei Yang, Yuexin Ma

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.CV

2407.08942 2024-07-15 cs.IR cs.AI 79%

A Neural Matrix Decomposition Recommender System Model based on the Multimodal Large Language Model

Ao Xiang, Bingjie Huang, Xinyu Guo, Haowei Yang, Tianyao Zheng

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

2407.04587 2024-07-08 cs.LG cs.CV 79%

Multimodal Classification via Modal-Aware Interactive Enhancement

Qing-Yuan Jiang, Zhouyang Chi, Yang Yang

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

2402.11943 2024-06-24 cs.CL 79%

LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation

Keyang Xuan, Li Yi, Fan Yang, Ruochen Wu, Yi R. Fung, Heng Ji

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CL

2402.07398 2024-06-21 cs.AI 79%

VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization

Dongsheng Zhu, Xunzhu Tang, Weidong Han, Jinghui Lu, Yukun Zhao, Guoliang Xing, Junfeng Wang, Dawei Yin

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted to NAACL2024 main conference

2406.12908 2024-06-21 cs.LG cs.AI stat.ME stat.ML 79%

Rating Multi-Modal Time-Series Forecasting Models (MM-TSFM) for Robustness Through a Causal Lens

Kausik Lakkaraju, Rachneet Kaur, Zhen Zeng, Parisa Zehtabi, Sunandita Patra, Biplav Srivastava, Marco Valtorta

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.AI

2406.10295 2024-06-18 cs.CL cs.IR 79%

Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM)

Anjanava Biswas, Wrick Talukdar

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.CL

Comments 20 pages, 6 figures

Journal ref Journal of Artificial Intelligence Research: Vol. 4 (2024): No. 1, 176-195

2405.18732 2024-06-17 physics.geo-ph cs.AI cs.LG physics.app-ph 79%

Gemini & Physical World: Large Language Models Can Estimate the Intensity of Earthquake Shaking from Multi-Modal Social Media Posts

S. Mostafa Mousavi, Marc Stogaitis, Tajinder Gadh, Richard M Allen, Alexei Barski, Robert Bosch, Patrick Robertson, Nivetha Thiruverahan, Youngmin Cho, Aman Raj

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.AI

2403.14077 2024-06-12 cs.AI cs.CR 79%

Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics

Shan Jia, Reilin Lyu, Kangran Zhao, Yize Chen, Zhiyuan Yan, Yan Ju, Chuanbo Hu, Xin Li, Baoyuan Wu, Siwei Lyu

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

2404.18975 2024-06-11 cs.LG cs.AI 79%

M3H: Multimodal Multitask Machine Learning for Healthcare

Dimitris Bertsimas, Yu Ma

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

2406.03207 2024-06-06 cs.CV cs.CE 79%

Identification of Stone Deterioration Patterns with Large Multimodal Models

Daniele Corradetti, Jose Delgado Rodrigues

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages, 5 figures, submitted to Journal of Cultural Heritage

2406.01302 2024-06-06 cs.CV 79%

Pulmonary Embolism Mortality Prediction Using Multimodal Learning Based on Computed Tomography Angiography and Clinical Data

Zhusi Zhong, Helen Zhang, Fayez H. Fayad, Andrew C. Lancaster, John Sollee, Shreyas Kulkarni, Cheng Ting Lin, Jie Li, Xinbo Gao, Scott Collins, Colin Greineder, Sun H. Ahn, Harrison X. Bai, Zhicheng Jiao, Michael K. Atalay

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CV

2406.01032 2024-06-04 cs.LG cs.AI 79%

LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning

Junjie Xu, Zongyu Wu, Minhua Lin, Xiang Zhang, Suhang Wang

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

2406.00735 2024-06-04 q-bio.BM cs.AI cs.LG 79%

Full-Atom Peptide Design based on Multi-modal Flow Matching

Jiahan Li, Chaoran Cheng, Zuofan Wu, Ruihan Guo, Shitong Luo, Zhizhou Ren, Jian Peng, Jianzhu Ma

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.AI

Comments ICML 2024

2405.18655 2024-05-30 cs.LG cs.AI q-bio.GN 79%

CAVACHON: a hierarchical variational autoencoder to integrate multi-modal single-cell data

Ping-Han Hsieh, Ru-Xiu Hsiao, Katalin Ferenc, Anthony Mathelier, Rebekka Burkholz, Chien-Yu Chen, Geir Kjetil Sandve, Tatiana Belova, Marieke Lydia Kuijjer

专题命中其他多模态：multi-modal(title,abstract)；分类 cs.AI

2401.13601 2024-05-29 cs.CL 79%

MM-LLMs: Recent Advances in MultiModal Large Language Models

Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, Dong Yu

专题命中其他多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by ACL2024 (findings)

2405.13426 2024-05-24 cs.HC cs.AI 79%

A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Models

Li Fan, Lee Ching-Hung, Han Su, Feng Shanshan, Jiang Zhuoxuan, Sun Zhu

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

Comments 14 pages, journal paper

2405.13099 2024-05-24 cs.AI cs.SI 79%

The Role of Emotions in Informational Support Question-Response Pairs in Online Health Communities: A Multimodal Deep Learning Approach

Mohsen Jozani, Jason A. Williams, Ahmed Aleroud, Sarbottam Bhagat

专题命中其他多模态：multimodal(title,abstract)；分类 cs.AI

Comments 37 pages, 15 figures

1. 其他多模态 4868 篇

Reliable Multi-modal Medical Image-to-image Translation Independent of Pixel-wise Aligned Data

Multi-modal Intermediate Feature Interaction AutoEncoder for Overall Survival Prediction of Esophageal Squamous Cell Cancer

D-RMGPT: Robot-assisted collaborative tasks driven by large multimodal models

Enhancing Cross-Modal Medical Image Segmentation through Compositionality

Leveraging Weak Cross-Modal Guidance for Coherence Modelling via Iterative Learning

Chain of Visual Perception: Harnessing Multimodal Large Language Models for Zero-shot Camouflaged Object Detection

Sparse vs Contiguous Adversarial Pixel Perturbations in Multimodal Models: An Empirical Analysis

Chameleon: Images Are What You Need For Multimodal Learning Robust To Missing Modalities

Decoupling Common and Unique Representations for Multimodal Self-supervised Learning

On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning

Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals

LabelDistill: Label-guided Cross-modal Knowledge Distillation for Camera-based 3D Object Detection

WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language

A Neural Matrix Decomposition Recommender System Model based on the Multimodal Large Language Model

Multimodal Classification via Modal-Aware Interactive Enhancement

LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation

VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization

Rating Multi-Modal Time-Series Forecasting Models (MM-TSFM) for Robustness Through a Causal Lens

Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM)

Gemini & Physical World: Large Language Models Can Estimate the Intensity of Earthquake Shaking from Multi-Modal Social Media Posts

Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics

M3H: Multimodal Multitask Machine Learning for Healthcare

Identification of Stone Deterioration Patterns with Large Multimodal Models

Pulmonary Embolism Mortality Prediction Using Multimodal Learning Based on Computed Tomography Angiography and Clinical Data

LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning

Full-Atom Peptide Design based on Multi-modal Flow Matching

CAVACHON: a hierarchical variational autoencoder to integrate multi-modal single-cell data

MM-LLMs: Recent Advances in MultiModal Large Language Models

A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Models

The Role of Emotions in Informational Support Question-Response Pairs in Online Health Communities: A Multimodal Deep Learning Approach