多模态大模型 - arXivDaily 专题

2503.14908 2025-03-20 cs.GR cs.AI cs.CV 73%

POSTA: A Go-to Framework for Customized Artistic Poster Generation

Haoyu Chen, Xiaojie Xu, Wenbo Li, Jingjing Ren, Tian Ye, Songhua Liu, Ying-Cong Chen, Lei Zhu, Xinchao Wang

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2025

2308.09599 2025-02-26 cs.CV cs.MM 73%

Language-Guided Diffusion Model for Visual Grounding

Sijia Chen, Baochun Li

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 20 pages, 16 figures

2412.15220 2024-12-23 cs.MM cs.SD eess.AS 73%

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

Haohe Liu, Gael Le Lan, Xinhao Mei, Zhaoheng Ni, Anurag Kumar, Varun Nagaraja, Wenwu Wang, Mark D. Plumbley, Yangyang Shi, Vikas Chandra

专题命中多模态生成：multi-modal(abstract)；audio-visual(abstract)；分类 cs.MM、eess.AS

2411.19220 2024-12-02 cs.CV cs.MM 73%

Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection

Tsun-Hin Cheung, Ka-Chun Fung, Songjiang Lai, Kwan-Ho Lin, Vincent Ng, Kin-Man Lam

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Comments Accepted to APSIPA ASC 2024

2411.17066 2024-11-27 cs.CV cs.CL cs.SC 73%

Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models

Colin Conwell, Rupert Tawiah-Quashie, Tomer Ullman

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

2411.15453 2024-11-26 cs.CV cs.AI 73%

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

Te Yang, Jian Jia, Xiangyu Zhu, Weisong Zhao, Bo Wang, Yanhua Cheng, Yan Li, Shengyuan Liu, Quan Chen, Peng Jiang, Kun Gai, Zhen Lei

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2406.04485 2024-11-12 cs.AI cs.CV 73%

GenAI Arena: An Open Evaluation Platform for Generative Models

Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, Wenhu Chen

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages,7 figures

Journal ref NeurIPS 2024

2411.01805 2024-11-05 cs.SD cs.MM eess.AS 73%

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence

Fuming You, Minghui Fang, Li Tang, Rongjie Huang, Yongqi Wang, Zhou Zhao

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.MM、eess.AS

Comments NeurIPS 2024

2409.17508 2024-11-04 cs.CV cs.AI cs.LG 73%

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

Xun Zhu, Ying Hu, Fanbin Mo, Miao Li, Ji Wu

专题命中多模态生成：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2410.03334 2024-10-07 cs.CV cs.AI 73%

An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation

Ahmed Abdulaal, Hugo Fry, Nina Montaña-Brown, Ayodeji Ijishakin, Jack Gao, Stephanie Hyland, Daniel C. Alexander, Daniel C. Castro

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

2409.06945 2024-09-12 cs.CV cs.AI 73%

FSMDet: Vision-guided feature diffusion for fully sparse 3D detector

Tianran Liu, Morteza Mousa Pasandi, Robert Laganiere

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by European Conference on Computer Vision (ECCV) 2024 workshop on VCAD

2404.14715 2024-07-23 cs.CV cs.CL 73%

FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, Jiebo Luo

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments ECCV 2024

2406.08478 2024-06-19 cs.CV cs.CL 73%

What If We Recaption Billions of Web Images with LLaMA-3?

Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, Cihang Xie

专题命中多模态生成：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments First five authors contributed equally

2406.07502 2024-06-12 cs.CV cs.CL 73%

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, Tong Zhang

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

2403.07750 2024-06-10 cs.CV cs.AI 73%

Synth$^2$: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings

Sahand Sharifzadeh, Christos Kaplanis, Shreya Pathak, Dharshan Kumaran, Anastasija Ilic, Jovana Mitrovic, Charles Blundell, Andrea Banino

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 6 figures

2405.15439 2024-05-27 cs.CV cs.AI 73%

Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer

Zichen Geng, Caren Han, Zeeshan Hayder, Jian Liu, Mubarak Shah, Ajmal Mian

专题命中多模态生成：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2402.05195 2024-04-11 cs.CV cs.CL 73%

$λ$-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space

Maitreya Patel, Sangmin Jung, Chitta Baral, Yezhou Yang

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://eclipse-t2i.github.io/Lambda-ECLIPSE/

2404.03590 2024-04-05 cs.CV cs.AI 73%

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization

Kailin Li, Jingbo Wang, Lixin Yang, Cewu Lu, Bo Dai

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2304.14178 2024-04-01 cs.CL cs.CV cs.LG 73%

mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality

Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qi Qian, Ji Zhang, Fei Huang, Jingren Zhou

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Working in Process

2308.09911 2024-03-29 cs.CV cs.MM 73%

Noisy-Correspondence Learning for Text-to-Image Person Re-identification

Yang Qin, Yingke Chen, Dezhong Peng, Xi Peng, Joey Tianyi Zhou, Peng Hu

专题命中多模态生成：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

2312.07424 2024-02-27 cs.LG cs.AI cs.CV 73%

How Well Does GPT-4V(ision) Adapt to Distribution Shifts? A Preliminary Investigation

Zhongyi Han, Guanglin Zhou, Rundong He, Jindong Wang, Tailin Wu, Yilong Yin, Salman Khan, Lina Yao, Tongliang Liu, Kun Zhang

专题命中多模态生成：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments added the investigation of Gemini. 66 pages, 41 figures

2305.13208 2024-01-24 cs.CV cs.AI 73%

Iterative Adversarial Attack on Image-guided Story Ending Generation

Youze Wang, Wenbo Hu, Richang Hong

专题命中多模态生成：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2401.02147 2024-01-05 cs.CL cs.CV 73%

Exploring Boundary of GPT-4V on Marine Analysis: A Preliminary Case Study

Ziqiang Zheng, Yiwei Chen, Jipeng Zhang, Tuan-Anh Vu, Huimin Zeng, Yue Him Wong Tim, Sai-Kit Yeung

专题命中多模态生成：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 51 pages, 36 figures, Repository: https://github.com/hkust-vgd/Marine_GPT-4V_Eval

2309.17421 2023-10-12 cs.CV cs.CL 73%

The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

专题命中多模态生成：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.CL

2306.11504 2023-06-21 cs.GR cs.CV cs.SD eess.AS 73%

Align, Adapt and Inject: Sound-guided Unified Image Generation

Yue Yang, Kaipeng Zhang, Yuying Ge, Wenqi Shao, Zeyue Xue, Yu Qiao, Ping Luo

专题命中多模态生成：multi-modal(abstract)；audio-visual(abstract)；分类 cs.CV、eess.AS

Comments Tech Report

2305.18752 2023-05-31 cs.CV cs.CL 73%

GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, Ying Shan

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

2303.12208 2023-03-23 cs.CV cs.CL cs.LG 73%

MAGVLT: Masked Generative Vision-and-Language Transformer

Sungwoong Kim, Daejin Jo, Donghoon Lee, Jongmin Kim

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments CVPR 2023

2209.14491 2022-11-23 cs.CV cs.AI cs.LG 73%

Re-Imagen: Retrieval-Augmented Text-to-Image Generator

Wenhu Chen, Hexiang Hu, Chitwan Saharia, William W. Cohen

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 9 pages

2112.01455 2022-05-05 cs.CV cs.AI cs.GR cs.LG 73%

Zero-Shot Text-Guided Object Generation with Dream Fields

Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, Ben Poole

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2022. 13 pages. Website: https://ajayj.com/dreamfields

2104.08910 2021-04-20 cs.CV cs.MM 73%

Towards Open-World Text-Guided Face Image Generation and Manipulation

Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.MM

Comments arXiv admin note: substantial text overlap with arXiv:2012.03308

1. 多模态生成 4951 篇

POSTA: A Go-to Framework for Customized Artistic Poster Generation

Language-Guided Diffusion Model for Visual Grounding

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection

Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

GenAI Arena: An Open Evaluation Platform for Generative Models

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation

FSMDet: Vision-guided feature diffusion for fully sparse 3D detector

FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

What If We Recaption Billions of Web Images with LLaMA-3?

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Synth$^2$: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings

Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer

$λ$-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization

mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality

Noisy-Correspondence Learning for Text-to-Image Person Re-identification

How Well Does GPT-4V(ision) Adapt to Distribution Shifts? A Preliminary Investigation

Iterative Adversarial Attack on Image-guided Story Ending Generation

Exploring Boundary of GPT-4V on Marine Analysis: A Preliminary Case Study

The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

Align, Adapt and Inject: Sound-guided Unified Image Generation

GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

MAGVLT: Masked Generative Vision-and-Language Transformer

Re-Imagen: Retrieval-Augmented Text-to-Image Generator

Zero-Shot Text-Guided Object Generation with Dream Fields

Towards Open-World Text-Guided Face Image Generation and Manipulation