ModelUniform Diffusion Language Model / 均匀扩散语言模型

UDLM

A variant of diffusion language models that applies uniform noising and denoising to tokens at arbitrary positions in a sequence, which is more difficult to train but yields stronger downstream reasoning ability.

Timeline (last 90 days)

Oct 22

UDLM must denoise from potentially contaminated context during training, whereas MDLM always predicts from clean context, so UDLM consumes more FLOPs than MDLM.

Unverified50%

Oct 22

UDLM performs better than MDLM on downstream reasoning tasks.

Unverified50%

All Facts (2)

Unverified

UDLM must denoise from potentially contaminated context during training, whereas MDLM always predicts from clean context, so UDLM consumes more FLOPs than MDLM.

50%Unverified

UDLM performs better than MDLM on downstream reasoning tasks.

50%

Source Articles

Looped Uniform Diffusion Language Models: 3.3x Reduction in Training FLOPsOct 22