多模态大模型 - arXivDaily 专题

2412.18390 2024-12-30 cs.CV cs.AI cs.LG cs.MM 67%

RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction

Xiaoping Wu, Jie Hu, Xiaoming Wei

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 8 pages

2412.12121 2024-12-18 cs.DL cs.AI cs.CL cs.CV cs.LG 67%

NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?

Christoph Leiter, Jonas Belouadi, Yanran Chen, Ran Zhang, Daniil Larionov, Aida Kostikova, Steffen Eger

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2412.07801 2024-12-12 cs.CV cs.AI cs.CL 67%

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

Jiali Chen, Xusen Hei, Yuqi Xue, Yuancheng Wei, Jiayuan Xie, Yi Cai, Qing Li

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by ACM MM 2024

2403.13802 2024-11-26 cs.CV cs.AI cs.CL cs.LG 67%

ZigMa: A DiT-style Zigzag Mamba Diffusion Model

Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Schusterbauer, Björn Ommer

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024 Project Page: https://taohu.me/zigma/

2411.15113 2024-11-25 cs.CV cs.AI cs.CL cs.LG 67%

Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

Samarth N Ramesh, Zhixue Zhao

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

2411.09955 2024-11-22 cs.CV cs.AI cs.HC cs.LG cs.MM 67%

Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era

Thanh Tam Nguyen, Zhao Ren, Trinh Pham, Thanh Trung Huynh, Phi Le Nguyen, Hongzhi Yin, Quoc Viet Hung Nguyen

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Fixed a serious error in author information

2411.09933 2024-11-18 cs.CV cs.AI cs.CL cs.NE 67%

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

Kaito Baba, Ryota Yagi, Junichiro Takahashi, Risa Kishikawa, Satoshi Kodera

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by NeurIPS'24 Workshop on AIM-FM: Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond

2410.20359 2024-11-04 cs.SD cs.AI cs.CV cs.GR eess.AS 67%

Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Gaoge Han, Jifeng Ning, Wei Liu

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、eess.AS

Comments Accepted by WACV 2025 (Round 1)

2404.04251 2024-11-01 cs.CV cs.AI cs.CL 67%

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi, Yujie Lu, Aditya Sharma, William Yang Wang

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024 Spotlight

2401.10712 2024-10-15 cs.CV cs.AI cs.CL 67%

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Haibo Wang, Weifeng Ge

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by ECCV'24

2406.00093 2024-10-04 cs.CV cs.AI cs.GR cs.LG cs.MM 67%

Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data

Zeyi Sun, Tong Wu, Pan Zhang, Yuhang Zang, Xiaoyi Dong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Project Page: https://sunzey.github.io/Bootstrap3D/

2311.16201 2024-09-26 cs.CV cs.AI cs.CL cs.LG 67%

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation

Yuhui Zhang, Brandon McKinzie, Zhe Gan, Vaishaal Shankar, Alexander Toshev

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Published at EMNLP 2024 Main Conference

2310.12128 2024-07-16 cs.CV cs.AI cs.CL cs.LG 67%

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning

Abhay Zala, Han Lin, Jaemin Cho, Mohit Bansal

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments COLM 2024; Project page: https://diagrammerGPT.github.io/

2407.03188 2024-07-12 cs.SD cs.AI cs.MM eess.AS 67%

MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generation

Zihao Wang, Haoxuan Liu, Jiaxing Yu, Tao Zhang, Yan Liu, Kejun Zhang

专题命中多模态生成：cross-modal(abstract)；分类 cs.AI、cs.MM、eess.AS

Comments 19 pages, 5 figures

2407.07771 2024-07-11 cs.CL cs.CV cs.MM 67%

Multi-task Prompt Words Learning for Social Media Content Generation

Haochen Xue, Chong Zhang, Chengzhi Liu, Fangyu Wu, Xiaobo Jin

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 8 pages, 5 figures

Journal ref International Joint Conference on Neural Networks 2024

2402.13254 2024-06-13 cs.CV cs.AI cs.CL cs.LG 67%

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples

Jianrui Zhang, Mu Cai, Tengyang Xie, Yong Jae Lee

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 15 pages, 6 figures, 12 tables, Project Page: https://countercurate.github.io/

2405.16277 2024-06-04 cs.CL cs.AI cs.CV cs.LG 67%

Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge

Brendan Park, Madeline Janecek, Naser Ezzati-Jivan, Yifeng Li, Ali Emami

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 9 pages (excluding references), accepted to ACL 2024 Main Conference

2305.12328 2024-05-30 cs.CV cs.AI cs.MM 67%

InstructVid2Vid: Controllable Video Editing with Natural Language Instructions

Bosheng Qin, Juncheng Li, Siliang Tang, Tat-Seng Chua, Yueting Zhuang

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ICME 2024

2404.18928 2024-04-30 cs.CV cs.AI cs.CL cs.GR cs.LG 67%

Stylus: Automatic Adapter Selection for Diffusion Models

Michael Luo, Justin Wong, Brandon Trabucco, Yanping Huang, Joseph E. Gonzalez, Zhifeng Chen, Ruslan Salakhutdinov, Ion Stoica

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Website: https://stylus-diffusion.github.io

2404.13899 2024-04-23 cs.CL cs.AI cs.MM 67%

Towards Better Text-to-Image Generation Alignment via Attention Modulation

Yihang Wu, Xiao Cao, Kaixin Li, Zitan Chen, Haonan Wang, Lei Meng, Zhiyong Huang

专题命中多模态生成：image-text(abstract)；分类 cs.CL、cs.AI、cs.MM

2312.02439 2024-04-23 cs.AI cs.CL cs.CV 67%

Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation

Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, Pan Zhou

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical report

2404.10763 2024-04-17 cs.AI cs.CL cs.CV 67%

LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?

Yuchi Wang, Shuhuai Ren, Rundong Gao, Linli Yao, Qingyan Guo, Kaikai An, Jianhong Bai, Xu Sun

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

2312.04552 2024-04-16 cs.CV cs.AI cs.LG cs.MM 67%

Generating Illustrated Instructions

Sachit Menon, Ishan Misra, Rohit Girdhar

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to CVPR 2024. Project website: http://facebookresearch.github.io/IllustratedInstructions. Code reproduction: https://github.com/sachit-menon/generating-illustrated-instructions-reproduction

2404.07206 2024-04-11 cs.CV cs.AI cs.GR cs.LG cs.MM 67%

GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models

Zewei Zhang, Huan Liu, Jun Chen, Xiangyu Xu

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

2403.18814 2024-03-28 cs.CV cs.AI cs.CL 67%

Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models

Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, Jiaya Jia

专题命中多模态生成：any-to-any(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Code and models are available at https://github.com/dvlab-research/MiniGemini

2403.06952 2024-03-12 cs.CV cs.AI cs.CL cs.LG 67%

SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data

Jialu Li, Jaemin Cho, Yi-Lin Sung, Jaehong Yoon, Mohit Bansal

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments First two authors contributed equally; Project website: https://selma-t2i.github.io/

2402.14623 2024-02-23 cs.RO cs.AI cs.CL cs.CV 67%

RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation

Junting Chen, Yao Mu, Qiaojun Yu, Tianming Wei, Silang Wu, Zhecheng Yuan, Zhixuan Liang, Chao Yang, Kaipeng Zhang, Wenqi Shao, Yu Qiao, Huazhe Xu, Mingyu Ding, Ping Luo

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 10 pages of main paper, 4 pages of appendix; 10 figures in main paper, 3 figures in appendix

2209.10134 2024-02-20 cs.MM cs.CL cs.CV 67%

Recipe Generation from Unsegmented Cooking Videos

Taichi Nishimura, Atsushi Hashimoto, Yoshitaka Ushiku, Hirotaka Kameko, Shinsuke Mori

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Accepted at ACM TOMM; ACM Transactions on Multimedia Computing, Communications, and Applications

2305.19216 2024-02-16 cs.CL cs.AI cs.CV cs.LG 67%

Translation-Enhanced Multilingual Text-to-Image Generation

Yaoyiran Li, Ching-Yun Chang, Stephen Rawls, Ivan Vulić, Anna Korhonen

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL 2023 (Main)

Journal ref Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pages 9174-9193

2401.01044 2024-01-03 cs.SD cs.AI cs.CL eess.AS 67%

Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation

Jinlong Xue, Yayue Deng, Yingming Gao, Ya Li

专题命中多模态生成：cross-modal(abstract)；分类 cs.CL、cs.AI、eess.AS

Comments Demo and implementation at https://auffusion.github.io

1. 多模态生成 4955 篇

RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction

NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

ZigMa: A DiT-style Zigzag Mamba Diffusion Model

Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning

MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generation

Multi-task Prompt Words Learning for Social Media Content Generation

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples

Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge

InstructVid2Vid: Controllable Video Editing with Natural Language Instructions

Stylus: Automatic Adapter Selection for Diffusion Models

Towards Better Text-to-Image Generation Alignment via Attention Modulation

Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation

LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?

Generating Illustrated Instructions

GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models

Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models

SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data

RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation

Recipe Generation from Unsegmented Cooking Videos

Translation-Enhanced Multilingual Text-to-Image Generation

Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation