多模态大模型

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation

Ruikai Cui, Weizhe Liu, Weixuan Sun, Senbo Wang, Taizhang Shang, Yang Li, Xibin Song, Han Yan, Zhennan Wu, Shenzhou Chen, Hongdong Li, Pan Ji

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024, project page: https://weizheliu.github.io/NeuSDFusion/

DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents

Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat, Karsten Kreis

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments project page: https://research.nvidia.com/labs/lpr/disco-diff

Revision Matters: Generative Design Guided by Revision Edits

Tao Li, Chin-Yi Cheng, Amber Xie, Gang Li, Yang Li

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models

Yichen Sun, Zhixuan Chu, Zhan Qin, Kui Ren

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

COT Flow: Learning Optimal-Transport Image Sampling and Editing by Contrastive Pairs

Xinrui Zu, Qian Tao

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Understanding Visual Concepts Across Models

Brandon Trabucco, Max Gurinas, Kyle Doherty, Ruslan Salakhutdinov

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Official code at: https://github.com/visual-words/visual-words

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Sambaran Bandyopadhyay, Himanshu Maheshwari, Anandhavelu Natarajan, Apoorv Saxena

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL、cs.AI

VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification

Jianmeng Liu, Yichen Liu, Yuyao Zhang, Zeyuan Meng, Yu-Wing Tai, Chi-Keung Tang

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 27pages, 16 figures

Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes

Hmrishav Bandyopadhyay, Subhadeep Koley, Ayan Das, Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Yi-Zhe Song

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024, Project Page: https://hmrishavbandy.github.io/doodle23d/

TexIm FAST: Text-to-Image Representation for Semantic Similarity Evaluation using Transformers

Wazib Ansar, Saptarsi Goswami, Amlan Chakrabarti

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 33 figures

The Good, The Bad, and Why: Unveiling Emotions in Generative AI

Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Xinyi Wang, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, Xing Xie

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments International Conference on Machine Learning (ICML) 2024; an extension to EmotionPrompt (arXiv:2307.11760)

VideoPoet: A Large Language Model for Zero-Shot Video Generation

Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, Krishna Somandepalli, Hassan Akbari, Yair Alon, Yong Cheng, Josh Dillon, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, Mikhail Sirotenko, Kihyuk Sohn, Xuan Yang, Hartwig Adam, Ming-Hsuan Yang, Irfan Essa, Huisheng Wang, David A. Ross, Bryan Seybold, Lu Jiang

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments To appear at ICML 2024; Project page: http://sites.research.google/videopoet/

Contextualized Diffusion Models for Text-Guided Image and Video Generation

Ling Yang, Zhilong Zhang, Zhaochen Yu, Jingwei Liu, Minkai Xu, Stefano Ermon, Bin Cui

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2024. Project: https://github.com/YangLing0818/ContextDiff

Intelligent Text-Conditioned Music Generation

Zhouyao Xie, Nikhil Yadala, Xinyi Chen, Jing Xi Liu

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.MM、eess.AS

You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet

Zhen Qin, Yuxin Mao, Xuyang Shen, Dong Li, Jing Zhang, Yuchao Dai, Yiran Zhong

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Technical report. Yiran Zhong is the corresponding author. The code is available at https://github.com/OpenNLPLab/LightNet

Faithful Chart Summarization with ChaTS-Pi

Syrine Krichene, Francesco Piccinno, Fangyu Liu, Julian Martin Eisenschlos

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments To be published in the proceedings of the 2024 Annual Meeting of the Association for Computational Linguistics

ProtT3: Protein-to-Text Generation for Text-based Protein Understanding

Zhiyuan Liu, An Zhang, Hao Fei, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CL、cs.MM

Comments ACL 2024, 9 pages

MMFusion: Multi-modality Diffusion Model for Lymph Node Metastasis Diagnosis in Esophageal Cancer

Chengyu Wu, Chengkai Wang, Yaqi Wang, Huiyu Zhou, Yatao Zhang, Qifeng Wang, Shuai Wang

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Early accepted to MICCAI 2024 (6/6/5)

FIRE: Food Image to REcipe generation

Prateek Chhikara, Dhiraj Chaurasia, Yifan Jiang, Omkar Masur, Filip Ilievski

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Published at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) -- 2024

Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective

Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao-Yong Wei, Hui Liu, Jiliang Tang, Qing Li

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE TKDE. Our implementation is available at: https://github.com/phenixace/MolReGPT

Discrepancy-based Diffusion Models for Lesion Detection in Brain MRI

Keqiang Fan, Xiaohao Cai, Mahesan Niranjan

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

From ChatGPT, DALL-E 3 to Sora: How has Generative AI Changed Digital Humanities Research and Services?

Jiangfeng Liu, Ziyi Wang, Jing Xie, Lei Pei

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 3 figures

A Systematic Review of Deep Learning-based Research on Radiology Report Generation

Chang Liu, Yuanhe Tian, Yan Song

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 26 pages, 6 figures

TAVGBench: Benchmarking Text to Audible-Video Generation

Yuxin Mao, Xuyang Shen, Jing Zhang, Zhen Qin, Jinxing Zhou, Mochu Xiang, Yiran Zhong, Yuchao Dai

专题命中 多模态生成 ：audio-visual(abstract)；分类 cs.CV、cs.MM

Comments Technical Report. Project page:https://github.com/OpenNLPLab/TAVGBench

EdgeFusion: On-Device Text-to-Image Generation

Thibault Castells, Hyoung-Kyu Song, Tairen Piao, Shinkook Choi, Bo-Kyeong Kim, Hanyoung Yim, Changgwun Lee, Jae Gon Kim, Tae-Ho Kim

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 4 pages, accepted to CVPR24 First Workshop on Efficient and On-Device Generation (EDGE)

ClickDiffusion: Harnessing LLMs for Interactive Precise Image Editing

Alec Helbling, Seongmin Lee, Polo Chau

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2402.07925

Opportunities and challenges in the application of large artificial intelligence models in radiology

Liangrui Pan, Zhenyu Zhao, Ying Lu, Kewei Tang, Liyong Fu, Qingchun Liang, Shaoliang Peng

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation

Dongchan Min, Minyoung Song, Eunji Ko, Sung Ju Hwang

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、eess.AS

GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models

Emilio Ferrara

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted in: Journal of Computational Social Science

Journal ref J Comput Soc Sc (2024)

Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation

Zhiwei Zhang, Yuliang Liu

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments TMLR, Survey Certification