多模态大模型 - arXivDaily 专题

2406.18247 2024-06-27 eess.IV cs.CV cs.LG 79%

Generative artificial intelligence in ophthalmology: multimodal retinal images for the diagnosis of Alzheimer's disease with convolutional neural networks

I. R. Slootweg, M. Thach, K. R. Curro-Tafili, F. D. Verbraak, F. H. Bouwman, Y. A. L. Pijnenburg, J. F. Boer, J. H. P. de Kwisthout, L. Bagheriye, P. J. González

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2406.14555 2024-06-21 cs.CV 79%

A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models

Xincheng Shuai, Henghui Ding, Xingjun Ma, Rongcheng Tu, Yu-Gang Jiang, Dacheng Tao

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Project Page: https://github.com/xinchengshuai/Awesome-Image-Editing

2406.13150 2024-06-21 eess.IV cs.CV 79%

MCAD: Multi-modal Conditioned Adversarial Diffusion Model for High-Quality PET Image Reconstruction

Jiaqi Cui, Xinyi Zeng, Pinxian Zeng, Bo Liu, Xi Wu, Jiliu Zhou, Yan Wang

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments Early accepted by MICCAI2024

2406.10561 2024-06-18 cs.CL 79%

We Care: Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic Response Generation

Palash Moon, Pushpak Bhattacharyya

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

2403.00822 2024-06-18 cs.IR cs.AI 79%

InteraRec: Screenshot Based Recommendations Using Multimodal Large Language Models

Saketh Reddy Karra, Theja Tulabandhula

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

2406.09162 2024-06-14 cs.CV 79%

EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts

Yucheng Han, Rui Wang, Chi Zhang, Juntao Hu, Pei Cheng, Bin Fu, Hanwang Zhang

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments https://tencentqqgylab.github.io/EMMA

2406.09003 2024-06-14 cs.CV cs.LG 79%

Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generation

Lincan Cai, Shuang Li, Wenxuan Ma, Jingxuan Kang, Binhui Xie, Zixun Sun, Chengwei Zhu

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

2404.06665 2024-06-14 cs.CV 79%

Deep Generative Data Assimilation in Multimodal Setting

Yongquan Qu, Juan Nathaniel, Shuolin Li, Pierre Gentine

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Best Student Paper Award @ CVPR2024 EarthVision

2406.05325 2024-06-11 eess.AS cs.SD 79%

LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance

Shihao Chen, Yu Gu, Jie Zhang, Na Li, Rilin Chen, Liping Chen, Lirong Dai

专题命中多模态生成：any-to-any(title,abstract)；分类 eess.AS

Comments Accepted by Interspeech 2024

2406.01334 2024-06-04 cs.CV 79%

HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Models

Mengcheng Li, Hongwen Zhang, Yuxiang Zhang, Ruizhi Shao, Tao Yu, Yebin Liu

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments accepted in CVPR2024, project page: https://dw1010.github.io/project/HHMR/HHMR.html

2405.16501 2024-05-28 cs.CV 79%

User-Friendly Customized Generation with Multi-Modal Prompts

Linhao Zhong, Yan Hong, Wentao Chen, Binglin Zhou, Yiyi Zhang, Jianfu Zhang, Liqing Zhang

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 8 figures

2405.15738 2024-05-27 cs.CV 79%

ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models

Chunjiang Ge, Sijie Cheng, Ziming Wang, Jiale Yuan, Yuan Gao, Jun Song, Shiji Song, Gao Huang, Bo Zheng

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments 17 pages

2402.12741 2024-05-27 cs.CV 79%

MuLan: Multimodal-LLM Agent for Progressive and Interactive Multi-Object Diffusion

Sen Li, Ruochen Wang, Cho-Jui Hsieh, Minhao Cheng, Tianyi Zhou

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Added the application to human-agent interaction; added discussion with concurrent work

2405.04834 2024-05-24 cs.CV 79%

FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation

Xuehai He, Jian Zheng, Jacob Zhiyuan Fang, Robinson Piramuthu, Mohit Bansal, Vicente Ordonez, Gunnar A Sigurdsson, Nanyun Peng, Xin Eric Wang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2404.01655 2024-05-21 cs.CV 79%

FashionEngine: Interactive 3D Human Generation and Editing via Multimodal Controls

Tao Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Project Page: https://taohuumd.github.io/projects/FashionEngine

2312.02233 2024-05-13 cs.CV 79%

MedXChat: A Unified Multimodal Large Language Model Framework towards CXRs Understanding and Generation

Ling Yang, Zhanyu Wang, Zhenghao Chen, Xinyu Liang, Luping Zhou

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2312.13286 2024-05-09 cs.CV 79%

Generative Multimodal Models are In-Context Learners

Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Zhengxiong Luo, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, Xinlong Wang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2024. Project page: https://baaivision.github.io/emu2

2405.04356 2024-05-08 cs.CV 79%

Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation

Jihyun Kim, Changjae Oh, Hoseok Do, Soohyun Kim, Kwanghoon Sohn

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2405.02911 2024-05-07 cs.CV 79%

Multimodal Sense-Informed Prediction of 3D Human Motions

Zhenyu Lou, Qiongjie Cui, Haofan Wang, Xu Tang, Hong Zhou

专题命中多模态生成：multimodal(title)；multi-modal(abstract)；分类 cs.CV

2312.14385 2024-05-07 cs.DC cs.LG cs.MM 79%

Generative AI Beyond LLMs: System Implications of Multi-Modal Generation

Alicia Golden, Samuel Hsia, Fei Sun, Bilge Acun, Basil Hosmer, Yejin Lee, Zachary DeVito, Jeff Johnson, Gu-Yeon Wei, David Brooks, Carole-Jean Wu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.MM

Comments Published at 2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)

2404.19444 2024-05-03 cs.CV 79%

AnomalyXFusion: Multi-modal Anomaly Synthesis with Diffusion

Jie Hu, Yawen Huang, Yilin Lu, Guoyang Xie, Guannan Jiang, Yefeng Zheng, Zhichao Lu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2404.19175 2024-05-01 cs.CL 79%

Game-MUG: Multimodal Oriented Game Situation Understanding and Commentary Generation Dataset

Zhihao Zhang, Feiqi Cao, Yingbin Mo, Yiran Zhang, Josiah Poon, Caren Han

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

2404.14604 2024-04-29 cs.CL 79%

Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training

Mengzhao Jia, Zhihan Zhang, Wenhao Yu, Fangkai Jiao, Meng Jiang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

2404.16678 2024-04-26 cs.CV 79%

Multimodal Semantic-Aware Automatic Colorization with Diffusion Prior

Han Wang, Xinning Chai, Yiwen Wang, Yuhong Zhang, Rong Xie, Li Song

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2404.15014 2024-04-24 cs.CV 79%

OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving

Guoqing Wang, Zhongdao Wang, Pin Tang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, Chao Ma

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2404.14567 2024-04-24 cs.CL 79%

WangLab at MEDIQA-M3G 2024: Multimodal Medical Answer Generation using Large Language Models

Ronald Xie, Steven Palayew, Augustin Toma, Gary Bader, Bo Wang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

2307.14367 2024-04-23 q-bio.QM cs.CL cs.LG 79%

Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformers

Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, Michalis Vazirgiannis

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

Journal ref Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), 10757-10765 (2024)

2404.01284 2024-04-02 cs.CV 79%

Large Motion Model for Unified Multi-Modal Motion Generation

Mingyuan Zhang, Daisheng Jin, Chenyang Gu, Fangzhou Hong, Zhongang Cai, Jingfang Huang, Chongzhi Zhang, Xinying Guo, Lei Yang, Ying He, Ziwei Liu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments Homepage: https://mingyuan-zhang.github.io/projects/LMM.html

2403.14828 2024-03-26 cs.CV 79%

Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing

Alberto Baldrati, Davide Morelli, Marcella Cornia, Marco Bertini, Rita Cucchiara

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2403.11700 2024-03-25 cs.MM 79%

Virbo: Multimodal Multilingual Avatar Video Generation in Digital Marketing

Juan Zhang, Jiahao Chen, Cheng Wang, Zhiwang Yu, Tangquan Qi, Can Liu, Di Wu

专题命中多模态生成：multimodal(title,abstract)；分类 cs.MM

1. 多模态生成 4946 篇

Generative artificial intelligence in ophthalmology: multimodal retinal images for the diagnosis of Alzheimer's disease with convolutional neural networks

A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models

MCAD: Multi-modal Conditioned Adversarial Diffusion Model for High-Quality PET Image Reconstruction

We Care: Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic Response Generation

InteraRec: Screenshot Based Recommendations Using Multimodal Large Language Models

EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts

Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generation

Deep Generative Data Assimilation in Multimodal Setting

LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance

HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Models

User-Friendly Customized Generation with Multi-Modal Prompts

ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models

MuLan: Multimodal-LLM Agent for Progressive and Interactive Multi-Object Diffusion

FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation

FashionEngine: Interactive 3D Human Generation and Editing via Multimodal Controls

MedXChat: A Unified Multimodal Large Language Model Framework towards CXRs Understanding and Generation

Generative Multimodal Models are In-Context Learners

Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation

Multimodal Sense-Informed Prediction of 3D Human Motions

Generative AI Beyond LLMs: System Implications of Multi-Modal Generation

AnomalyXFusion: Multi-modal Anomaly Synthesis with Diffusion

Game-MUG: Multimodal Oriented Game Situation Understanding and Commentary Generation Dataset

Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training

Multimodal Semantic-Aware Automatic Colorization with Diffusion Prior

OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving

WangLab at MEDIQA-M3G 2024: Multimodal Medical Answer Generation using Large Language Models

Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformers

Large Motion Model for Unified Multi-Modal Motion Generation

Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing

Virbo: Multimodal Multilingual Avatar Video Generation in Digital Marketing