多模态大模型

MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaboration

Yanbo Ding, Shaobin Zhuang, Kunchang Li, Zhengrong Yue, Yu Qiao, Yali Wang

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments AAAI 2025

MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments

Yuqi Tong, Yue Qiu, Ruiyang Li, Shi Qiu, Pheng-Ann Heng

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments IEEE AIxVR 2025

Multimodal Latent Language Modeling with Next-Token Diffusion

Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, Furu Wei

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Parametric-ControlNet: Multimodal Control in Foundation Models for Precise Engineering Design Synthesis

Rui Zhou, Yanxia Zhang, Chenyang Yuan, Frank Permenter, Nikos Arechiga, Matt Klenk, Faez Ahmed

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Examining Multimodal Gender and Content Bias in ChatGPT-4o

Roberto Balestri

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 4 figures, 3 tables. Conference: "14th International Conference on Artificial Intelligence, Soft Computing and Applications (AIAA 2024), London, 23-24 November 2024" It will be published in the proceedings "David C. Wyld et al. (Eds): IoTE, CNDC, DSA, AIAA, NLPTA, DPPR - 2024"

XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation

Ziyi Wang, Yanbo Wang, Xumin Yu, Jie Zhou, Jiwen Lu

专题命中 多模态生成 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024

Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Models

Zhen Zeng, Leijiang Gu, Xun Yang, Zhangling Duan, Zenglin Shi, Meng Wang

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices

Xudong Lu, Yinghao Chen, Cheng Chen, Hui Tan, Boheng Chen, Yina Xie, Rui Hu, Guanxin Tan, Renshou Wu, Yan Hu, Yi Zeng, Lei Wu, Liuyang Bian, Zhaoxiong Wang, Long Liu, Yanzhou Yang, Han Xiao, Aojun Zhou, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 21 pages

MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation

Jialin Luo, Yuanzhi Wang, Ziqi Gu, Yide Qiu, Shuaizhen Yao, Fuyun Wang, Chunyan Xu, Wenhua Zhang, Dan Wang, Zhen Cui

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024

MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency

Junzhe Zhang, Huixuan Zhang, Xunjian Yin, Baizhou Huang, Xu Zhang, Xinyu Hu, Xiaojun Wan

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

PixelBytes: Catching Unified Representation for Multimodal Generation

Fabien Furfaro

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 4 figures

GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs

Pu Hua, Minghuan Liu, Annabella Macaluso, Yunfeng Lin, Weinan Zhang, Huazhe Xu, Lirui Wang

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments CoRL 2024. Project website: https://gensim2.github.io/

Synthetic Multimodal Question Generation

Ian Wu, Sravan Jayanthi, Vijay Viswanathan, Simon Rosenberg, Sina Pakazad, Tongshuang Wu, Graham Neubig

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Findings; Camera Ready

FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instruction

Runze He, Kai Ma, Linjiang Huang, Shaofei Huang, Jialin Gao, Xiaoming Wei, Jiao Dai, Jizhong Han, Si Liu

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 14 figures, project website: https://freeedit.github.io/

Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Research

Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, W. Jim Zheng, Hongyu Zhao

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages

Journal ref EMNLP 2024

EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models

Shangyu Xing, Fei Zhao, Zhen Wu, Tuo An, Weihao Chen, Chunhui Li, Jianbing Zhang, Xinyu Dai

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

MUMU: Bootstrapping Multimodal Image Generation from Text-to-Image Data

William Berman, Alexander Peysakhovich

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

KOSMOS-2.5: A Multimodal Literate Model

Tengchao Lv, Yupan Huang, Jingye Chen, Yuzhong Zhao, Yilin Jia, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generation

Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Kang Zhang, Yu-Jung Heo, Du-Seong Chang, Chang D. Yoo

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments ECCV 2024

From text to multimodal: a survey of adversarial example generation in question answering systems

Gulsum Yigit, Mehmet Fatih Amasyali

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments published in Knowledge and Information Systems

Journal ref Knowl Inf Syst (2024)

Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports

Yutong Zhang, Yi Pan, Tianyang Zhong, Peixin Dong, Kangni Xie, Yuxiao Liu, Hanqi Jiang, Zhengliang Liu, Shijie Zhao, Tuo Zhang, Xi Jiang, Dinggang Shen, Tianming Liu, Xin Zhang

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition

Zirun Guo, Tao Jin, Zhou Zhao

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACL 2024 Main

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?

Zhaorun Chen, Yichao Du, Zichen Wen, Yiyang Zhou, Chenhang Cui, Zhenzhen Weng, Haoqin Tu, Chaoqi Wang, Zhengwei Tong, Qinglan Huang, Canyu Chen, Qinghao Ye, Zhihong Zhu, Yuqing Zhang, Jiawei Zhou, Zhuokai Zhao, Rafael Rafailov, Chelsea Finn, Huaxiu Yao

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 42 pages, 13 figures, 33 tables

GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generation

Shihao Cai, Keqin Bao, Hangyu Guo, Jizhi Zhang, Jun Song, Bo Zheng

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

GenMM: Geometrically and Temporally Consistent Multimodal Data Generation for Video and LiDAR

Bharat Singh, Viveka Kulharia, Luyu Yang, Avinash Ravichandran, Ambrish Tyagi, Ashish Shrivastava

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities

Roman Bachmann, Oğuzhan Fatih Kar, David Mizrahi, Ali Garjani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, Amir Zamir

专题命中 多模态生成 ：any-to-any(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project page at 4m.epfl.ch

TRINS: Towards Multimodal Language Models that Can Read

Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

PostDoc: Generating Poster from a Long Multimodal Document Using Deep Submodular Optimization

Vijay Jaisankar, Sambaran Bandyopadhyay, Kalp Vyas, Varre Chaitanya, Shwetha Somasundaram

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective

Zihao Yue, Liang Zhang, Qin Jin

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACL 2024

CamViG: Camera Aware Image-to-Video Generation with Multimodal Transformers

Andrew Marmon, Grant Schindler, José Lezama, Dan Kondratyuk, Bryan Seybold, Irfan Essa

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI