Continuous Diffusion Language Model
A language model paradigm that maps discrete tokens into a continuous embedding space before applying the diffusion process, performing noise addition and denoising in that continuous space before decoding back to discrete tokens. Representative works include Diffusion-LM, MDLM, and Plaid; the paradigm enables continuous methods such as flow matching to be applied directly to language generation.
Timeline (last 90 days)
The optimal inference configuration for continuous diffusion language models is not universal but should be adjusted to the task objective: pursuing precise reasoning favors deterministic refinement, while pursuing generation diversity requires preserving randomness.
A recent paper proposes directly coupling the continuous latent space framework to continuous diffusion language models (dLLMs), replacing random mapping with a learned language distribution to generate more natural adversarial prompts.
Continuous diffusion language models (dLLMs) are a class of models that apply diffusion models to text generation, generating text in a continuous latent space through a denoising process, and their latent-space-to-text mapping is learned rather than random.
All Facts (3)
The optimal inference configuration for continuous diffusion language models is not universal but should be adjusted to the task objective: pursuing precise reasoning favors deterministic refinement, while pursuing generation diversity requires preserving randomness.
50%UnverifiedA recent paper proposes directly coupling the continuous latent space framework to continuous diffusion language models (dLLMs), replacing random mapping with a learned language distribution to generate more natural adversarial prompts.
50%UnverifiedContinuous diffusion language models (dLLMs) are a class of models that apply diffusion models to text generation, generating text in a continuous latent space through a denoising process, and their latent-space-to-text mapping is learned rather than random.
50%