多模态大模型 - arXivDaily 专题

2303.16203 2023-09-14 cs.LG cs.AI cs.CV cs.NE cs.RO 62%

Your Diffusion Model is Secretly a Zero-Shot Classifier

Alexander C. Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, Deepak Pathak

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

Comments In ICCV 2023. Website at https://diffusion-classifier.github.io/

2303.15233 2023-09-07 cs.CV cs.AI cs.LG 62%

Text-to-Image Diffusion Models are Zero-Shot Classifiers

Kevin Clark, Priyank Jaini

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

2306.16805 2023-09-04 cs.CV cs.CL cs.LG 62%

CLIPAG: Towards Generator-Free Text-to-Image Generation

Roy Ganz, Michael Elad

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL

2308.06721 2023-08-15 cs.CV cs.AI 62%

IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, Wei Yang

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2308.04249 2023-08-09 cs.CV cs.AI 62%

MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion

Yizhuo Lu, Changde Du, Qiongyi zhou, Dianpeng Wang, Huiguang He

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2303.14139

2308.01147 2023-08-03 cs.CV cs.MM eess.IV 62%

Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation

Guojin Zhong, Jin Yuan, Pan Wang, Kailun Yang, Weili Guan, Zhiyong Li

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM MM 2023. The code will be released at https://github.com/zgj77/FSACDM

2307.16013 2023-08-01 cs.AI cs.CL cs.DB 62%

Marrying Dialogue Systems with Data Visualization: Interactive Data Visualization Generation from Natural Language Conversations

Yuanfeng Song, Xuefang Zhao, Raymond Chi-Wing Wong

专题命中多模态生成：multi-modal(abstract)；分类 cs.CL、cs.AI

2305.14720 2023-06-23 cs.CV cs.AI 62%

BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing

Dongxu Li, Junnan Li, Steven C. H. Hoi

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2303.05334 2023-06-22 cs.CV cs.AI q-bio.NC 62%

Natural scene reconstruction from fMRI signals using generative latent diffusion

Furkan Ozcelik, Rufin VanRullen

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2305.14598 2023-05-25 cs.CV cs.MM 62%

Vision + Language Applications: A Survey

Yutong Zhou, Nobutaka Shimada

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by GCV @CVPR2023

2305.11517 2023-05-22 cs.CL cs.AI 62%

DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusion

Chao-Hong Tan, Jia-Chen Gu, Zhen-Hua Ling

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Work in Progress

2305.11116 2023-05-19 cs.CV cs.CL 62%

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, William Yang Wang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL

2305.04757 2023-05-19 cs.CL cs.AI cs.LG 62%

Augmented Large Language Models with Parametric Knowledge Guiding

Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

2305.09662 2023-05-17 cs.CV cs.AI 62%

Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

Samaneh Azadi, Akbar Shah, Thomas Hayes, Devi Parikh, Sonal Gupta

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2304.07410

2304.11829 2023-04-26 cs.CV cs.AI 62%

Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation

Zeyu Lu, Chengyue Wu, Xinyuan Chen, Yaohui Wang, Lei Bai, Yu Qiao, Xihui Liu

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

2304.01603 2023-04-05 cs.CV cs.AI 62%

Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA

Yongxin Zhu, Zhen Liu, Yukang Liang, Xin Li, Hao Liu, Changcun Bao, Linli Xu

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments accepted in AAAI 2023

2304.01515 2023-04-05 cs.LG cs.CL cs.CV 62%

Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models

Jaewoong Lee, Sangwon Jang, Jaehyeong Jo, Jaehong Yoon, Yunji Kim, Jin-Hwa Kim, Jung-Woo Ha, Sung Ju Hwang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL

2210.15257 2023-03-29 cs.CV cs.AI 62%

ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts

Zhida Feng, Zhenyu Zhang, Xintong Yu, Yewei Fang, Lanxin Li, Xuyi Chen, Yuxiang Lu, Jiaxiang Liu, Weichong Yin, Shikun Feng, Yu Sun, Li Chen, Hao Tian, Hua Wu, Haifeng Wang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2023, highlight. Project page: https://wenxin.baidu.com/ernie-vilg

2303.14139 2023-03-27 cs.CV cs.AI 62%

MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion

Yizhuo Lu, Changde Du, Dianpeng Wang, Huiguang He

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

2205.12502 2023-03-03 cs.CV cs.CL cs.LG 62%

The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training

Gi-Cheon Kang, Sungdong Kim, Jin-Hwa Kim, Donghyun Kwak, Byoung-Tak Zhang

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL

Comments CVPR 2023

2302.12192 2023-02-24 cs.LG cs.AI cs.CV 62%

Aligning Text-to-Image Models using Human Feedback

Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Shixiang Shane Gu

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

2301.12661 2023-01-31 cs.SD cs.LG cs.MM eess.AS 62%

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, Zhou Zhao

专题命中多模态生成：multimodal(abstract)；分类 cs.MM、eess.AS

Comments Audio samples are available at https://Text-to-Audio.github.io

2301.02110 2023-01-06 cs.CV cs.AI cs.LG 62%

FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversion

Martin Pernuš, Clinton Fookes, Vitomir Štruc, Simon Dobrišek

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

2301.00493 2023-01-03 cs.CV cs.AI cs.LG cs.RO 62%

Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting

Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, James Hays

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks

2210.10047 2022-12-19 cs.RO cs.AI cs.CV cs.LG 62%

From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data

Zichen Jeff Cui, Yibin Wang, Nur Muhammad Mahi Shafiullah, Lerrel Pinto

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Code and data available at: https://play-to-policy.github.io; (fixed metadata author name format)

2211.01761 2022-11-04 cs.CL cs.AI 62%

PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning

Zifeng Wang, Jimeng Sun

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2022

2210.14124 2022-10-26 cs.CV cs.AI 62%

Lafite2: Few-shot Text-to-Image Generation

Yufan Zhou, Chunyuan Li, Changyou Chen, Jianfeng Gao, Jinhui Xu

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

2210.12997 2022-10-25 cs.CL cs.CV 62%

Are Current Decoding Strategies Capable of Facing the Challenges of Visual Dialogue?

Amit Kumar Chaudhary, Alex J. Lucassen, Ioanna Tsani, Alberto Testoni

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at INLG 2022

2210.11522 2022-10-24 cs.CV cs.AI cs.LG 62%

Composing Ensembles of Pre-trained Models via Iterative Consensus

Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Igor Mordatch

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2210.02399 2022-10-06 cs.CV cs.AI 62%

Phenaki: Variable Length Video Generation From Open Domain Textual Description

Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, Dumitru Erhan

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

1. 多模态生成 4959 篇

Your Diffusion Model is Secretly a Zero-Shot Classifier

Text-to-Image Diffusion Models are Zero-Shot Classifiers

CLIPAG: Towards Generator-Free Text-to-Image Generation

IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion

Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation

Marrying Dialogue Systems with Data Visualization: Interactive Data Visualization Generation from Natural Language Conversations

BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing

Natural scene reconstruction from fMRI signals using generative latent diffusion

Vision + Language Applications: A Survey

DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusion

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Augmented Large Language Models with Parametric Knowledge Guiding

Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation

Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA

Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models

ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts

MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion

The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training

Aligning Text-to-Image Models using Human Feedback

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversion

Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting

From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data

PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning

Lafite2: Few-shot Text-to-Image Generation

Are Current Decoding Strategies Capable of Facing the Challenges of Visual Dialogue?

Composing Ensembles of Pre-trained Models via Iterative Consensus

Phenaki: Variable Length Video Generation From Open Domain Textual Description