Salesforce - arXivDaily 大厂专区

2607.13431 2026-07-16 cs.LG cs.AI cs.CL 新提交

Discrete Diffusion Models: A Unified Framework from Tokenization to Generation

离散扩散模型：从词元化到生成的统一框架

Ye Yuan, Weien Li, Rui Song, Zeyu Li, Haochen Liu, Xiangyu Kong, Zixuan Dong, Linfeng Du, Zipeng Sun, Weixu Zhang, Jiaxin Huang, Changjiang Han, Yonghan Yang, Zichen Zhao, Xiuyuan Hu, Haolun Wu, Yankai Chen, Fengran Mo, Jikun Kang, Bowei He, Philip S. Yu, Xue Liu

发表机构 * McGill University（麦吉尔大学）； Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）； University of Cambridge（剑桥大学）； University of Toronto（多伦多大学）； MBZUAI - Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）； Tsinghua University（清华大学）； Rochester Institute of Technology（罗彻斯特理工学院）； Salesforce（Salesforce公司）； University of Illinois Chicago（伊利诺伊大学芝加哥分校）

AI总结研究离散扩散模型，引入统一框架从离散状态空间构建审视该模型，让现有公式成为共同设计空间实例，揭示训练、推理等方面权衡，为未来研究提供方向。

详情

AI中文摘要

离散去噪扩散模型（DDMs）最近成为离散数据自回归建模的有力替代方案，具有并行生成和迭代全局细化能力。与连续扩散不同，离散扩散模型的状态空间由离散状态空间的构建方式决定。本文引入统一概念框架，通过构建底层离散状态空间来审视离散扩散模型。在此框架下，现有公式成为共同设计空间的不同实例，还揭示了训练目标、推理算法等方面的常见权衡，为未来研究指明方向。

英文摘要

Discrete denoising diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete data, offering parallel generation and iterative global refinement capabilities. Unlike continuous diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.

URL PDF HTML ☆