多模态大模型 - arXivDaily 专题

2412.11849 2024-12-17 eess.IV cs.CV cs.LG 79%

Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI

Ramy A. Zeineldin, Franziska Mathis-Ullrich

专题命中多模态生成：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments Accepted at the MICCAI BraTS Challenge 2023

2411.01859 2024-12-17 eess.IV cs.CV 79%

A Novel Deep Learning Tractography Fiber Clustering Framework for Functionally Consistent White Matter Parcellation Using Multimodal Diffusion MRI and Functional MRI

Jin Wang, Bocheng Guo, Yijie Li, Junyi Wang, Yuqian Chen, Jarrett Rushmore, Nikos Makris, Yogesh Rathi, Lauren J O'Donnell, Fan Zhang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments 5 pages, 3 figures

2404.12154 2024-12-17 cs.CV 79%

StyleBooth: Image Style Editing with Multimodal Instruction

Zhen Han, Chaojie Mao, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2108.02160 2024-12-13 eess.IV cs.CV 79%

MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease

Apoorva Sikka, Skand Peri, Jitender Singh Virk, Usma Niyaz, Deepti R. Bathula

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments 24 pages

2412.07141 2024-12-11 cs.CV 79%

Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation

Qianhao Han, Junyi Liu, Zengchang Qin, Zheng Zheng

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted in IEEE Big Data 2024

2412.06840 2024-12-11 cs.LG cs.CV 79%

MDiFF: Exploiting Multimodal Score-based Diffusion Models for New Fashion Product Performance Forecasting

Andrea Avogaro, Luigi Capogrosso, Franco Fummi, Marco Cristani

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at the FashionAI workshop @ the European Conference on Computer Vision (ECCV) 2024. arXiv admin note: substantial text overlap with arXiv:2412.05566

2412.05605 2024-12-10 cs.CV 79%

RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation

Xiang Gao, Kai Lu

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

2412.05566 2024-12-10 cs.CV cs.LG 79%

Dif4FF: Leveraging Multimodal Diffusion Models and Graph Neural Networks for Accurate New Fashion Product Performance Forecasting

Andrea Avogaro, Luigi Capogrosso, Franco Fummi, Marco Cristani

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at the 27th International Conference on Pattern Recognition (ICPR 2024)

2411.00499 2024-12-09 cs.CV cs.ET cs.LG eess.SP 79%

Cross-modal semantic segmentation for indoor environmental perception using single-chip millimeter-wave radar raw data

Hairuo Hu, Haiyong Cong, Zhuyu Shao, Yubo Bi, Jinghao Liu

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

Comments 5291 words, 17 pages, 11 figures

2412.03809 2024-12-06 cs.CV 79%

EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM

Quang Nguyen, Truong Vu, Trong-Tung Nguyen, Yuxin Wen, Preston K Robinette, Taylor T Johnson, Tom Goldstein, Anh Tran, Khoi Nguyen

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2412.01407 2024-12-04 cs.CV 79%

HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving

Zehuan Wu, Jingcheng Ni, Xiaodong Wang, Yuxin Guo, Rui Chen, Lewei Lu, Jifeng Dai, Yuwen Xiong

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2412.00243 2024-12-03 cs.RO cs.AI 79%

Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model

Qiujing Lu, Meng Ma, Ximiao Dai, Xuanhan Wang, Shuo Feng

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

2411.18850 2024-12-02 cs.CV 79%

CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction

Lipeng Gu, Xuefeng Yan, Weiming Wang, Honghua Chen, Dingkun Zhu, Liangliang Nan, Mingqiang Wei

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2405.00448 2024-11-21 cs.CV 79%

MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generation

Xujie Zhang, Ente Lin, Xiu Li, Yuxuan Luo, Michael Kampffmeyer, Xin Dong, Xiaodan Liang

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2405.16273 2024-11-05 cs.CV 79%

M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation

Mingshuang Luo, Ruibing Hou, Zhuo Li, Hong Chang, Zimo Liu, Yaowei Wang, Shiguang Shan

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2024, 21 pages, 7 figures

2410.23905 2024-11-01 cs.CV 79%

Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model

Hao Zhang, Lei Cao, Jiayi Ma

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2402.12647 2024-10-28 cs.CV cs.RO 79%

DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation

Takuya Ikeda, Sergey Zakharov, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, Koichi Nishiwaki

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments 8 pages. 9 figures. This work has been submitted to the IEEE for possible publication

2405.16942 2024-10-24 eess.IV cs.CV 79%

PASTA: Pathology-Aware MRI to PET Cross-Modal Translation with Diffusion Models

Yitong Li, Igor Yakushev, Dennis M. Hedderich, Christian Wachinger

专题命中多模态生成：cross-modal(title)；multi-modal(abstract)；分类 cs.CV

Journal ref Medical Image Computing and Computer Assisted Intervention (MICCAI 2024)

2410.17032 2024-10-23 cs.AI 79%

Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups

Charvi Rastogi, Tian Huey Teh, Pushkar Mishra, Roma Patel, Zoe Ashwood, Aida Mostafazadeh Davani, Mark Diaz, Michela Paganini, Alicia Parrish, Ding Wang, Vinodkumar Prabhakaran, Lora Aroyo, Verena Rieser

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

Comments 20 pages, 7 figures

2410.16472 2024-10-23 cs.CL 79%

DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding

Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad I Morariu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

Comments EMNLP 2024 (Main)

2410.14089 2024-10-21 cs.CV 79%

MMAD-Purify: A Precision-Optimized Framework for Efficient and Scalable Multi-Modal Attacks

Xinxin Liu, Zhongliang Guo, Siyuan Huang, Chun Pong Lau

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2312.06738 2024-10-18 cs.CV 79%

InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following

Shufan Li, Harkanwar Singh, Aditya Grover

专题命中多模态生成：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments 25 pages, 19 figures

2410.11723 2024-10-16 cs.RO cs.AI math.OC 79%

Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers

Davide Celestini, Amirhossein Afsharrad, Daniele Gammelli, Tommaso Guffanti, Gioele Zardini, Sanjay Lall, Elisa Capello, Simone D'Amico, Marco Pavone

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

Comments 8 pages, 6 figures, submitted to 2025 American Control Conference (ACC)

2404.05465 2024-10-15 cs.CV cs.LG 79%

HAMMR: HierArchical MultiModal React agents for generic VQA

Lluis Castrejon, Thomas Mensink, Howard Zhou, Vittorio Ferrari, Andre Araujo, Jasper Uijlings

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2410.05143 2024-10-08 cs.CV 79%

Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Information

Timofey Efimov, Harry Dong, Megna Shah, Jeff Simmons, Sean Donegan, Yuejie Chi

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2410.04721 2024-10-08 cs.LG cs.CV 79%

ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction

Hyungjin Chung, Dohun Lee, Jong Chul Ye

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments 25 pages, 10 figures. Project page: https://acdc2025.github.io/

2403.07179 2024-10-04 cs.LG cs.CL q-bio.BM 79%

3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generation

Huaisheng Zhu, Teng Xiao, Vasant G Honavar

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CL

2401.10226 2024-10-02 cs.CV 79%

Towards Language-Driven Video Inpainting via Multimodal Large Language Models

Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, Chen Change Loy

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments CVPR-2024. Project Page: https://jianzongwu.github.io/projects/rovi

2311.00265 2024-10-02 eess.IV cs.CV 79%

Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study

Jonghun Kim, Hyunjin Park

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments 8 pages, 7 figures, WACV 2024 Accepted

2409.19862 2024-10-01 cs.LG cs.CV 79%

Learning Multimodal Latent Generative Models with Energy-Based Prior

Shiyu Yuan, Jiali Cui, Hanao Li, Tian Han

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments The 18th European Conference on Computer Vision ECCV 2024

1. 多模态生成 4946 篇

Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI

A Novel Deep Learning Tractography Fiber Clustering Framework for Functionally Consistent White Matter Parcellation Using Multimodal Diffusion MRI and Functional MRI

StyleBooth: Image Style Editing with Multimodal Instruction

MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease

Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation

MDiFF: Exploiting Multimodal Score-based Diffusion Models for New Fashion Product Performance Forecasting

RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation

Dif4FF: Leveraging Multimodal Diffusion Models and Graph Neural Networks for Accurate New Fashion Product Performance Forecasting

Cross-modal semantic segmentation for indoor environmental perception using single-chip millimeter-wave radar raw data

EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM

HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving

Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model

CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction

MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generation

M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation

Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model

DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation

PASTA: Pathology-Aware MRI to PET Cross-Modal Translation with Diffusion Models

Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups

DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding

MMAD-Purify: A Precision-Optimized Framework for Efficient and Scalable Multi-Modal Attacks

InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following

Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers

HAMMR: HierArchical MultiModal React agents for generic VQA

Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Information

ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction

3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generation

Towards Language-Driven Video Inpainting via Multimodal Large Language Models

Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study

Learning Multimodal Latent Generative Models with Energy-Based Prior