多模态大模型 - arXivDaily 专题

2503.06467 2025-03-11 cs.CV 83%

SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts

Shijia Zhao, Qiming Xia, Xusheng Guo, Pufan Zou, Maoji Zheng, Hai Wu, Chenglu Wen, Cheng Wang

专题命中多模态生成：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 11 pages, 3 figures

2405.12833 2025-03-07 cs.CV 83%

A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data

Xinyi Wang, Grazziela Figueredo, Ruizhe Li, Wei Emma Zhang, Weitong Chen, Xin Chen

专题命中多模态生成：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2404.14396 2025-03-04 cs.CV 83%

SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation

Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, Ying Shan

专题命中多模态生成：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments We added benchmark results (without updating models) and ablation study in this version. Project released at: https://github.com/AILab-CVC/SEED-X

2403.16188 2025-02-25 cs.CV 83%

Cross-domain Multi-modal Few-shot Object Detection via Rich Text

Zeyu Shangguan, Daniel Seita, Mohammad Rostami

专题命中多模态生成：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2408.16883 2025-02-04 cs.LG cs.CV 83%

Multimodal ELBO with Diffusion Decoders

Daniel Wesego, Pedram Rooshenas

专题命中多模态生成：multimodal(title,abstract)；any-to-any(abstract)；分类 cs.CV

2501.12327 2025-01-22 cs.CV 83%

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

Xianwei Zhuang, Yuxin Xie, Yufan Deng, Liming Liang, Jinghan Ru, Yuguo Yin, Yuexian Zou

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2412.12627 2025-01-07 cs.CL 83%

Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation

Andong Chen, Yuchen Song, Kehai Chen, Muyun Yang, Tiejun Zhao, Min Zhang

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

Comments Work in progress

2412.17077 2024-12-24 cs.AI 83%

SubstationAI: Multimodal Large Model-Based Approaches for Analyzing Substation Equipment Faults

Jinzhi Wang, Qinfeng Song, Lidong Qian, Haozhou Li, Qinke Peng, Jiangbo Zhang

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI

2412.12982 2024-12-18 eess.IV cs.CV 83%

Stable Diffusion is a Natural Cross-Modal Decoder for Layered AI-generated Image Compression

Ruijie Chen, Qi Mao, Zhengxue Cheng

专题命中多模态生成：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2412.09618 2024-12-13 cs.CV 83%

EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM

Zhuofan Zong, Dongzhi Jiang, Bingqi Ma, Guanglu Song, Hao Shao, Dazhong Shen, Yu Liu, Hongsheng Li

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Tech report

2407.00978 2024-12-10 cs.AI cs.LG 83%

Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach

Cheng Su, Jinbo Wen, Jiawen Kang, Yonghua Wang, Yuanjia Su, Hudan Pan, Zishao Zhong, M. Shamim Hossain

专题命中多模态生成：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.AI

Comments 13 pages, 7 figures

2406.01059 2024-12-02 cs.CV 83%

VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model

Jinze Yang, Haoran Wang, Zining Zhu, Chenglong Liu, Meng Wymond Wu, Mingming Sun

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by ACCV-2025, Our source code is available at: https://github.com/ucasyjz/VIP, 15 pages

2411.09449 2024-11-15 cs.CV 83%

Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models

Chutian Meng, Fan Ma, Jiaxu Miao, Chi Zhang, Yi Yang, Yueting Zhuang

专题命中多模态生成：multimodal(title)；MLLM(abstract)；cross-modal(abstract)；分类 cs.CV

2409.19872 2024-11-01 cs.CV 83%

Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration

Kaihang Pan, Zhaoyu Fan, Juncheng Li, Qifan Yu, Hao Fei, Siliang Tang, Richang Hong, Hanwang Zhang, Qianru Sun

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024 (Spotlight)

2407.05600 2024-10-29 cs.CV 83%

GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing

Zhenyu Wang, Aoxue Li, Zhenguo Li, Xihui Liu

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments NeurIPS 2024 Spotlight

2311.16511 2024-10-29 cs.CV 83%

GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation

Zhanyu Wang, Longyue Wang, Zhen Zhao, Minghao Wu, Chenyang Lyu, Huayang Li, Deng Cai, Luping Zhou, Shuming Shi, Zhaopeng Tu

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments ACM MM 2024, Oral

2407.15488 2024-10-22 cs.CV 83%

DiffX: Guide Your Layout to Cross-Modal Generative Modeling

Zeyu Wang, Jingyu Lin, Yifei Qian, Yi Huang, Shicen Tian, Bosong Chai, Juncan Deng, Qu Yang, Lan Du, Cunjian Chen, Kejie Huang

专题命中多模态生成：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.11403 2024-10-16 cs.LG cs.AI 83%

Enhancing Unimodal Latent Representations in Multimodal VAEs through Iterative Amortized Inference

Yuta Oshima, Masahiro Suzuki, Yutaka Matsuo

专题命中多模态生成：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

Comments 22 pages, 12 figures

2410.04932 2024-10-08 cs.CV 83%

OmniBooth: Learning Latent Control for Image Synthesis with Multi-modal Instruction

Leheng Li, Weichao Qiu, Xu Yan, Jing He, Kaiqiang Zhou, Yingjie Cai, Qing Lian, Bingbing Liu, Ying-Cong Chen

专题命中多模态生成：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.03787 2024-10-08 cs.CL cs.AI cs.CV cs.MM 83%

CalliffusionV2: Personalized Natural Calligraphy Generation with Flexible Multi-modal Control

Qisheng Liao, Liang Li, Yulang Fei, Gus Xia

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 11 pages, 7 figures

2410.01594 2024-10-03 cs.CV 83%

MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation

Mingzhen Sun, Weining Wang, Yanyuan Qiao, Jiahui Sun, Zihan Qin, Longteng Guo, Xinxin Zhu, Jing Liu

专题命中多模态生成：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024

2407.17571 2024-09-26 cs.CV 83%

Diffusion Models For Multi-Modal Generative Modeling

Changyou Chen, Han Ding, Bunyamin Sisman, Yi Xu, Ouye Xie, Benjamin Z. Yao, Son Dinh Tran, Belinda Zeng

专题命中多模态生成：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Published as a conference paper at ICLR 2024

2408.12141 2024-08-23 cs.CV 83%

TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model

Yuhao Wang, Chao Hao, Yawen Cui, Xinqi Su, Weicheng Xie, Tao Tan, Zitong Yu

专题命中多模态生成：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2408.10467 2024-08-21 cs.LG cs.CV 83%

Learning Multimodal Latent Space with EBM Prior and MCMC Inference

Shiyu Yuan, Carlo Lipizzi, Tian Han

专题命中多模态生成：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2408.08693 2024-08-19 cs.CL 83%

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm

Hongcheng Liu, Yusheng Liao, Siqv Ou, Yuhao Wang, Heyang Liu, Yanfeng Wang, Yu Wang

专题命中多模态生成：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CL

Comments 26 pages, 5 figures

2311.16500 2024-07-30 cs.CV 83%

LLMGA: Multimodal Large Language Model based Generation Assistant

Bin Xia, Shiyin Wang, Yingfan Tao, Yitong Wang, Jiaya Jia

专题命中多模态生成：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments The paper is accepted by ECCV2024. The code is available at https://github.com/dvlab-research/LLMGA

2403.05049 2024-07-22 cs.CV 83%

XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution

Yunpeng Qu, Kun Yuan, Kai Zhao, Qizhi Xie, Jinhua Hao, Ming Sun, Chao Zhou

专题命中多模态生成：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 19 pages, 7 figures; including supplementary material

2407.11315 2024-07-17 cs.AI 83%

COMET: "Cone of experience" enhanced large multimodal model for mathematical problem generation

Sannyuya Liu, Jintian Feng, Zongkai Yang, Yawei Luo, Qian Wan, Xiaoxuan Shen, Jianwen Sun

专题命中多模态生成：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

2407.07518 2024-07-11 cs.CV 83%

Multi-modal Crowd Counting via a Broker Modality

Haoliang Meng, Xiaopeng Hong, Chenhao Wang, Miao Shang, Wangmeng Zuo

专题命中多模态生成：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments This is the preprint version of the paper and supplemental material to appear in ECCV 2024. Please cite the final published version. Code is available at https://github.com/HenryCilence/Broker-Modality-Crowd-Counting

2407.05980 2024-07-09 cs.CV 83%

MMIS: Multimodal Dataset for Interior Scene Visual Generation and Recognition

Hozaifa Kassab, Ahmed Mahmoud, Mohamed Bahaa, Ammar Mohamed, Ali Hamdi

专题命中多模态生成：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

1. 多模态生成 4932 篇

SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts

A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data

SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation

Cross-domain Multi-modal Few-shot Object Detection via Rich Text

Multimodal ELBO with Diffusion Decoders

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation

SubstationAI: Multimodal Large Model-Based Approaches for Analyzing Substation Equipment Faults

Stable Diffusion is a Natural Cross-Modal Decoder for Layered AI-generated Image Compression

EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM

Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach

VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model

Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models

Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration

GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing

GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation

DiffX: Guide Your Layout to Cross-Modal Generative Modeling

Enhancing Unimodal Latent Representations in Multimodal VAEs through Iterative Amortized Inference

OmniBooth: Learning Latent Control for Image Synthesis with Multi-modal Instruction

CalliffusionV2: Personalized Natural Calligraphy Generation with Flexible Multi-modal Control

MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation

Diffusion Models For Multi-Modal Generative Modeling

TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model

Learning Multimodal Latent Space with EBM Prior and MCMC Inference

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm

LLMGA: Multimodal Large Language Model based Generation Assistant

XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution

COMET: "Cone of experience" enhanced large multimodal model for mathematical problem generation

Multi-modal Crowd Counting via a Broker Modality

MMIS: Multimodal Dataset for Interior Scene Visual Generation and Recognition