arXivDaily arXiv每日学术速递 周一至周五更新

AI 大模型

多模态大模型

跨文本、图像、视频、音频等模态的大模型与学习方法。

共收录 4955 信号源:cs.CV, cs.CL, cs.AI, cs.MM, eess.AS

1. 多模态生成 4955 篇

2405.04007 2024-05-08 cs.CV 70%

SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editing

Yuying Ge, Sijie Zhao, Chen Li, Yixiao Ge, Ying Shan

专题命中 多模态生成 :multimodal(abstract);MLLM(abstract);分类 cs.CV

Comments Technical Report; Dataset released in https://huggingface.co/datasets/AILab-CVC/SEED-Data-Edit

详情

展开后加载摘要…

URL PDF HTML 收藏
2404.13766 2024-04-23 cs.CV 70%

Object-Attribute Binding in Text-to-Image Generation: Evaluation and Control

Maria Mihaela Trusca, Wolf Nuyts, Jonathan Thomm, Robert Honig, Thomas Hofmann, Tinne Tuytelaars, Marie-Francine Moens

专题命中 多模态生成 :multimodal(abstract);image-text(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2404.09516 2024-04-16 cs.LG cs.AI cs.CL cs.CV cs.MM 70%

State Space Model for New-Generation Network Alternative to Transformers: A Survey

Xiao Wang, Shiao Wang, Yuhe Ding, Yuehang Li, Wentao Wu, Yao Rong, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, Ziwen Wang, Bo Jiang, Chenglong Li, Yaowei Wang, Yonghong Tian, Jin Tang

专题命中 多模态生成 :multi-modal(abstract);分类 cs.CV、cs.CL、cs.AI

Comments The First review of State Space Model (SSM)/Mamba and their applications in artificial intelligence, 33 pages

详情

展开后加载摘要…

URL PDF HTML 收藏
2312.13537 2023-12-22 cs.CV 70%

HyperEditor: Achieving Both Authenticity and Cross-Domain Capability in Image Editing via Hypernetworks

Hai Zhang, Chunwei Wu, Guitao Cao, Hailing Wang, Wenming Cao

专题命中 多模态生成 :cross-modal(abstract);image-text(abstract);分类 cs.CV

Comments Accepted by AAAI2024

详情

展开后加载摘要…

URL PDF HTML 收藏
2312.09251 2023-12-15 cs.CV 70%

VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation

Jinguo Zhu, Xiaohan Ding, Yixiao Ge, Yuying Ge, Sijie Zhao, Hengshuang Zhao, Xiaohua Wang, Ying Shan

专题命中 多模态生成 :multimodal(abstract);image-text(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2311.00571 2023-11-02 cs.CV cs.AI cs.CL cs.HC cs.MM 70%

LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing

Wei-Ge Chen, Irina Spiridonova, Jianwei Yang, Jianfeng Gao, Chunyuan Li

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.CL、cs.AI

Comments 31 pages, 22 figures, 30M PDF file size; Project Page: https://llava-vl.github.io/llava-interactive/

详情

展开后加载摘要…

URL PDF HTML 收藏
2310.11513 2023-10-19 cs.CV cs.LG 70%

GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment

Dhruba Ghosh, Hanna Hajishirzi, Ludwig Schmidt

专题命中 多模态生成 :multimodal(abstract);image-text(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2309.16206 2023-10-06 eess.IV cs.CV 70%

Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network

Qiankun Zuo, Junren Pan, Shuqiang Wang

专题命中 多模态生成 :multimodal(abstract);cross-modal(abstract);分类 cs.CV

Comments 10 pages

详情

展开后加载摘要…

URL PDF HTML 收藏
2309.00615 2023-09-04 cs.CV cs.AI cs.CL cs.LG cs.MM 70%

Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following

Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, Pheng-Ann Heng

专题命中 多模态生成 :multi-modal(abstract);分类 cs.CV、cs.CL、cs.AI

Comments Work in progress. Code is available at https://github.com/ZiyuGuo99/Point-Bind_Point-LLM

详情

展开后加载摘要…

URL PDF HTML 收藏
2307.08041 2023-08-15 cs.CV 70%

Planting a SEED of Vision in Large Language Model

Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, Ying Shan

专题命中 多模态生成 :multimodal(abstract);image-text(abstract);分类 cs.CV

Comments Technical Report; Project released at: https://github.com/AILab-CVC/SEED

详情

展开后加载摘要…

URL PDF HTML 收藏
2204.03039 2022-07-22 cs.CV 70%

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

Yilun Chen, Shijia Huang, Shu Liu, Bei Yu, Jiaya Jia

专题命中 多模态生成 :multi-modal(abstract);cross-modal(abstract);分类 cs.CV

Comments 13 pages

详情

展开后加载摘要…

URL PDF HTML 收藏
2204.02035 2022-04-06 cs.CV 70%

DT2I: Dense Text-to-Image Generation from Region Descriptions

Stanislav Frolov, Prateek Bansal, Jörn Hees, Andreas Dengel

专题命中 多模态生成 :multi-modal(abstract);image-text(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2111.13792 2022-03-25 cs.CV cs.LG 70%

LAFITE: Towards Language-Free Training for Text-to-Image Generation

Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, Tong Sun

专题命中 多模态生成 :multi-modal(abstract);image-text(abstract);分类 cs.CV

Comments Accepted by CVPR 2022, https://github.com/drboog/Lafite

详情

展开后加载摘要…

URL PDF HTML 收藏
2001.08779 2020-01-27 cs.CV cs.AI cs.CL cs.LG cs.MM 70%

Deep Bayesian Network for Visual Question Generation

Badri N. Patro, Vinod K. Kurmi, Sandeep Kumar, Vinay P. Namboodiri

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.CL、cs.AI

Comments WACV-2020 (Accepted)

详情

展开后加载摘要…

URL PDF HTML 收藏
1711.10485 2017-11-30 cs.CV 70%

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He

专题命中 多模态生成 :multimodal(abstract);image-text(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
1307.6549 2013-07-25 cs.CV cs.GR math.SP 70%

Making Laplacians commute

Michael M. Bronstein, Klaus Glashoff, Terry A. Loring

专题命中 多模态生成 :multimodal(abstract);multi-modal(abstract);分类 cs.CV

详情

展开后加载摘要…

URL PDF HTML 收藏
2305.14882 2024-04-16 cs.CL cs.AI cs.CV 69%

Dynamic Clue Bottlenecks: Towards Interpretable-by-Design Visual Question Answering

Xingyu Fu, Ben Zhou, Sihao Chen, Mark Yatskar, Dan Roth

专题命中 多模态生成 :multimodal(abstract,comments);分类 cs.CV、cs.CL、cs.AI

Comments Multimodal, Visual Question Answering, Vision and Language

详情

展开后加载摘要…

URL PDF HTML 收藏
2012.03308 2021-03-30 cs.CV cs.AI cs.MM 69%

TediGAN: Text-Guided Diverse Face Image Generation and Manipulation

Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu

专题命中 多模态生成 :multi-modal(abstract,comments);分类 cs.CV、cs.AI、cs.MM

Comments CVPR 2021. Code: https://github.com/weihaox/TediGAN Data: https://github.com/weihaox/Multi-Modal-CelebA-HQ Video: https://youtu.be/L8Na2f5viAM

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.18279 2026-08-20 physics.optics cs.LG 新提交 67%

A Comprehensive Review of Large Language Models for Nanophotonics: From Surrogate Modeling to Autonomous Design

面向纳米光子学的大语言模型综合综述:从代理建模到自主设计

Huanshu Zhang, Kegeng Tang, Lei Kang, Sawyer D. Campbell, Zihao Wang, Douglas H. Werner

专题命中 多模态生成 :multimodal(abstract);multimodal foundation model(abstract)

AI总结 该综述探讨大语言模型(LLMs)如何通过语义接口、代码生成及工具编排改进纳米光子学工作流,梳理相关方法的两类模式及跨学科应用,展望具备物理感知的多模态基础模型,推动AI从被动工具向主动科研合作者转变。

Comments Accepted for publication in Advanced Photonics

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.13560 2026-08-14 cs.CV cs.AI cs.CL 新提交 67%

AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

AutoDesign:面向长视距智能体设计的元工具优化

Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li

机构 * Meituan(美团) MBZUAI(Mohamed bin Zayed University of Artificial Intelligence) Huazhong University of Science and Technology(华中科技大学) Peking University(北京大学) Tsinghua University(清华大学) The Chinese University of Hong Kong(香港中文大学) Shanghai Jiao Tong University(上海交通大学)

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.CL、cs.AI

AI总结 AutoDesign是符合人类设计先验的元工具优化框架,以论文转海报生成任务为实例,在PosterBench上性能优于Claude Design,集成其学习的DesignHarness可提升代码智能体性能,且获人类最高偏好。

Comments Tech Report. Code at: https://github.com/Yaxin9Luo/AutoDesign

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.02833 2026-08-05 cs.CV cs.AI cs.CL 新提交 67%

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

CURV:通过课程可视化接地推理增强图表理解

Xuehang Guo, Pingyue Zhang, Ruiyi Zhang, Zhenhailong Wang, Hanrui Lyu, Heng Ji, Tong Sun, Qingyun Wang, Manling Li

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.CL、cs.AI

AI总结 针对多模态大语言模型视觉接地与推理不足的问题,提出CURV课程学习框架,结合CCQA数据集,在图表问答任务中实现显著性能提升并具备良好泛化性。

详情

展开后加载摘要…

URL PDF HTML 收藏
2608.02477 2026-08-04 cs.IR 新提交 67%

Unpaired Modality-Agnostic Generative Recommendation

非配对模态无关生成式推荐

Weihao Shen, Wei Chen, Fuwei Zhang, Meng Yuan, Yuqin Lan, Guojun Liu, Qingsong Hua, Wei Lin, Fuzhen Zhuang

专题命中 多模态生成 :multimodal(abstract);cross-modal(abstract)

AI总结 提出UnpairGR模型,从多类观测中学习统一语义ID空间,在三类基准数据集上验证其可提升完全与不完全观测下的推荐性能。

详情

展开后加载摘要…

URL PDF HTML 收藏
2602.12304 2026-07-24 cs.SD cs.AI cs.MM eess.AS 版本更新 67%

OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model

OmniCustom: 通过联合音视频生成模型实现同步音视频定制

Maomao Li, Zhen Li, Kaipeng Zhang, Guosheng Yin, Zhifeng Li, Dong Xu

机构 * The University of Hong Kong(香港大学) Shanda AI Research Tokyo(Shanda AI东京研究所) XIntelligence Technology Co., Limited(XIntelligence技术有限公司)

专题命中 多模态生成 :audio-visual(abstract);分类 cs.AI、cs.MM、eess.AS

AI总结 提出一种基于DiT的零样本音视频定制框架OmniCustom,通过参考图像和音频同步生成保持身份和音色一致性的视频,支持文本指定语音内容。

Comments code: https://github.com/OmniCustom-project/OmniCustom

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.19064 2026-07-23 cs.CV cs.AI cs.LG cs.MM eess.IV 版本更新 67%

Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

Mage-Flow:用于图像生成和编辑的高效原生分辨率基础模型

Xinjie Zhang, Peng Zhang, Shicheng Zheng, Jinghao Guo, Zhaoyang Jia, Yifei Shen, Xun Guo, Yuxuan Luo, Jiahao Li, Wenxuan Xie, Fanyi Pu, Xiaoyi Zhang, Kaichen Zhang, Zongyu Guo, Tianci Bi, Dongnan Gui, Zhening Liu, Zimo Wen, Zihan Zheng, Senqiao Yang, Xiao Li, Jinglu Wang, Bin Li, Yan Lu

机构 * Microsoft Mage Team(微软Mage团队)

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.AI、cs.MM

AI总结 研究针对大规模视觉生成器成本高的问题,提出Mage-Flow,由Mage-VAE和原生分辨率多模态扩散Transformer组成。通过协同设计实现高效文本到图像生成及编辑,开发完整模型家族,Turbo变体在高分辨率下生成和编辑高效,性能有竞争力。

详情

展开后加载摘要…

URL PDF HTML 收藏
2607.13188 2026-07-16 cs.LG 新提交 67%

Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes

并发图像理解与生成:自校正耦合马尔可夫跳跃过程

Minh-Quan Le, Armand Comas, Alexandros Lattas, Stylianos Moschoglou, Pedro Vélez, Amit Raj, Aaron Germuth, Thabo Beeler, Dimitris Samaras, Di Qiu

机构 * Stony Brook University(纽约州立大学石溪分校) Google DeepMind(谷歌DeepMind)

专题命中 多模态生成 :multimodal(abstract);cross-modal(abstract)

AI总结 研究针对人类认知中理解与生成的耦合循环,引入自校正耦合马尔可夫跳跃过程框架及$\texttt{CO}_\texttt{2}\texttt{Jump}$采样器,解决掩码扩散模型跨模态矛盾问题,创建多模态语料库,该方法在图像相关任务中性能优异,且性能随去噪步骤数提升。

Comments Project page: https://coupled-jump.github.io

详情

展开后加载摘要…

URL PDF HTML 收藏
2605.28198 2026-07-15 cs.LG 版本更新 67%

Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

层次化合成表格数据生成:一种自上而下与自下而上混合框架

Junfeng Nie, Alvin Jin, Xiaohui Chen

机构 * University of Southern California(南加州大学)

专题命中 多模态生成 :multimodal(abstract);cross-modal(abstract)

AI总结 提出一种层次化混合自上而下和自下而上(H-TDBU)框架,通过解耦语义结构与随机纹理,结合结构驱动的逻辑约束和轻量级表格生成器,在弱多模态金融基准上提升合成数据的语义一致性和统计保真度。

Comments Accepted as a poster at FMSD @ ICML 2026. 9 pages, 6 figures

详情

展开后加载摘要…

URL PDF HTML 收藏
2603.10791 2026-06-19 eess.IV 版本更新 67%

Semantic Satellite Communications for Synchronized Audiovisual Reconstruction

面向同步视听重建的语义卫星通信

Fangyu Liu, Peiwen Jiang, Wenjin Wang, Xiao Li, Shi Jin

专题命中 多模态生成 :multimodal(abstract);cross-modal(abstract)

AI总结 提出自适应多模态语义传输系统,通过双流生成架构和动态关键帧更新机制,在带宽受限的卫星场景下实现高质量同步视听重建,显著降低带宽消耗并提升鲁棒性。

详情

展开后加载摘要…

URL PDF HTML 收藏
2606.11210 2026-06-11 cs.CL cs.AI cs.MM 新提交 67%

T2MM: An LLM Supported Architecture For Inquiry-Based Modeling

T2MM:一种支持基于探究建模的LLM架构

John Kos, Rudra Singh, Ashok Goel

机构 * Georgia Institute of Technology(佐治亚理工学院)

专题命中 多模态生成 :multimodal(abstract);分类 cs.CL、cs.AI、cs.MM

AI总结 提出T2MM架构,利用LLM在生态建模软件VERA中生成交互式模型,优于全代码生成基线。

Comments 16 pages, 4 figures

详情

展开后加载摘要…

URL PDF HTML 收藏
2605.25531 2026-06-09 eess.SP 版本更新 67%

From Denoising to Decision Making: A Survey on Diffusion Model-Enabled Deep Reinforcement Learning for Wireless Networks

从去噪到决策:面向无线网络的扩散模型赋能深度强化学习综述

Nguyen Cong Luong, Zeping Sui, Jie Cao, Min Xu, Nguyen Duc Hai, Zhihao Dong, Nguyen Duc Duy Anh, Qiushi Zhao, Nguyen Quoc Khanh, Zhe Fu, Shaohan Feng, Bo Ma

专题命中 多模态生成 :multimodal(abstract);multi-modal(abstract)

AI总结 本文综述了扩散模型与深度强化学习结合的方法,通过捕捉无线资源管理中的复杂多模态动作结构,提升决策质量,并系统总结了其在移动边缘计算、无人机辅助、车联网、AIGC驱动系统、无线资源分配、物理层安全及机器人/无人机规划等领域的应用。

Comments 22 pages, 7 figures, Author list corrected

详情

展开后加载摘要…

URL PDF HTML 收藏
2601.15408 2026-06-09 cs.CV cs.AI cs.CL cs.LG 版本更新 67%

CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation

CURE:基于课程引导的多任务训练实现可靠的解剖学接地报告生成

Pablo Messina, Andrés Villa, Juan León Alcázar, Karen Sánchez, Carlos Hinojosa, Denis Parra, Álvaro Soto, Bernard Ghanem

机构 * Pontificia Universidad Católica de Chile(智利天主教大学) CENIA iHEALTH KAUST(科威特皇家科学与技术局)

专题命中 多模态生成 :multimodal(abstract);分类 cs.CV、cs.CL、cs.AI

AI总结 提出CURE框架,通过课程学习动态调整多任务训练,提升医学报告生成的视觉接地准确性和事实一致性,无需额外数据。

Comments 31 pages, 7 figures, accepted to CVPR 2026 (oral)

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 36279-36289

详情

展开后加载摘要…

URL PDF HTML 收藏