DiT
A diffusion model architecture proposed by Meta AI in 2023 that replaces the traditional U-Net backbone with a Vision Transformer. It exhibits strong scalability with large parameter counts and has been adopted by advanced video generation models such as Sora.
Core Facts
MiniMax H3 is a multimodal video generation model capable of simultaneously generating video frames and matching audio (including dialogue, sound effects, and ambient sound).
DiT was proposed by Peebles and Xie in 2022, introducing the Transformer architecture into the diffusion process.
H3 Minimax is Minimax's video generation engine, which adopts a Sparse Attention mechanism.
Timeline (last 90 days)
The DiT (Diffusion Transformer) toolkit is based on the low-end Transformer and is a core tool with a very high usage rate in the industry.
Open-source DiT implementations usually provide pretrained models and clear training scripts, making them suitable as an entry point for learning the code implementation of diffusion models.
DiT has become the base architecture of many advanced diffusion models (such as Sora).
The core idea of DiT is to use a Transformer to process sequences of image patches and to inject timestep and class conditions through adaptive layer normalization (adaLN).
Traditional diffusion models mostly use U-Net as the denoising network, whereas DiT replaces U-Net with a pure Transformer, splitting the image into a patch sequence as input and modeling global dependencies through self-attention.
The DiT source code structure is clean: one Patchify layer, several DiT Blocks (including adaptive layer normalization adaLN injecting timestep and class conditions), and one linear decoding layer, with none of the complex skip connections found in U-Net.
The recommended route for learning diffusion models from zero is a theory track (core mathematical derivations → forward/reverse processes → close reading of papers on acceleration, lightweighting, and structural optimization) and a practice track (lightweight toolkits such as DiT → reading the source code line by line → getting 2-3 core codebases running → understanding the essence of the algorithm → designing innovation points).
For beginners learning diffusion models, it is recommended to start with a lightweight DiT (Diffusion Transformer) toolkit first, because its source code is concise and clear, it comes with many ready-made open-source models built in, and it has strong adaptability.
Working through the DiT source code also lets you understand the diffusion model's training loop, noise schedule, sampler implementation, and conditioning injection mechanism along the way.
The core idea of DiT is to split the image into patches and treat them as a sequence input to the Transformer, and to inject timestep and class conditions through adaptive layer normalization (adaLN).
40 more timeline events
All Facts (20)
MiniMax H3 is a multimodal video generation model capable of simultaneously generating video frames and matching audio (including dialogue, sound effects, and ambient sound).
90%VerifiedDiT was proposed by Peebles and Xie in 2022, introducing the Transformer architecture into the diffusion process.
90%VerifiedH3 Minimax is Minimax's video generation engine, which adopts a Sparse Attention mechanism.
80%VerifiedSora adopts the DiT (Diffusion Transformer) architecture, compressing video into spatiotemporal patch sequences before feeding them into a Transformer, replacing the traditional U-Net with a Transformer as the denoising backbone network.
80%VerifiedThe Wan (万象) video generation model is based on the DiT (Diffusion Transformer) architecture.
80%VerifiedFLUX adopts a DiT (Diffusion Transformer) architecture, replacing the traditional U-Net structure with a pure Transformer.
80%VerifiedDiffusion Transformer (DiT) is a class of generative backbone networks that introduces the Transformer architecture into diffusion models, cutting noisy images into patches and linearly embedding them into token sequences, and using multi-layer Transformer blocks to predict noise or velocity, replacing the U-Net in traditional diffusion models.
80%VerifiedDiT is the foundational architecture of next-generation generative models such as Stable Diffusion 3 and FLUX.
75%VerifiedThe reverse process of diffusion models typically uses U-Net or DiT (Diffusion Transformer) architectures to predict the amount of noise to be removed at each step.
75%VerifiedDiffusion Transformer (DiT) is the dominant architecture in the current image generation field, used in models such as Stable Diffusion 3 and FLUX.
75%VerifiedThe denoising process of diffusion models typically requires tens to hundreds of iterations (called sampling steps), each of which requires a complete forward pass through the neural network.
75%VerifiedThe Diffusion Transformer (DiT) architecture was first proposed by Meta Research in 2022.
75%UnverifiedTransformer架构被引入视觉生成（如Vision Transformer、DiT架构），使模型能更好理解图像不同区域间的全局关系，适用于多模态场景中文本token和图像token的统一处理
88%UnverifiedThe core idea of DiT is to use a Transformer to process sequences of image patches and to inject timestep and class conditions through adaptive layer normalization (adaLN).
60%UnverifiedTraditional diffusion models mostly use U-Net as the denoising network, whereas DiT replaces U-Net with a pure Transformer, splitting the image into a patch sequence as input and modeling global dependencies through self-attention.
60%UnverifiedThe image generation field is undergoing an architectural paradigm shift from U-Net to Transformer; traditional diffusion models such as Stable Diffusion 1.x and 2.x primarily rely on a U-Net encoder-decoder structure.
60%UnverifiedOpenAI's Sora is widely believed to have adopted an architecture similar to DiT.
60%UnverifiedMainstream video generation models such as Sora and CogVideoX widely adopt the DiT or its variant architectures, marking the transition of video generation technology from the UNet era to the Transformer era.
60%UnverifiedThe DiT architecture replaces the traditional U-Net with a Transformer as the diffusion model backbone, enabling the capture of long-range cross-frame dependencies at the patch level.
60%UnverifiedOpenAI's Sora and Google's VideoPoet both adopt the technical approach of compressing video frames into discrete tokens followed by autoregressive prediction
60%