VAE

Stable Diffusion compresses the diffusion process from pixel space to latent space, implemented by a Variational Autoencoder (VAE).

Latent Diffusion通过VAE将图像从512×512×3的像素空间压缩到64×64×4的潜空间表示，数据量缩减约48倍

VAE (Variational Autoencoder) plays the role of image compression and decoding in the diffusion model pipeline, restoring latent space representations to pixel-level images.

Stable Diffusion compresses a 512×512×3 image into a 64×64×4 latent representation via VAE, achieving a compression ratio of 48×.

The Latent Diffusion Model was proposed by Rombach et al. in 2022, combining the diffusion process with the latent space of a VAE, reducing the computational dimensions from 512×512×3 to 64×64×4.

David Ha and Schmidhuber published the paper "World Models" in 2018, demonstrating how an agent can complete reinforcement learning training within dreams

The VAE (Variational Autoencoder) is responsible for encoding and decoding between latent space and pixel space, and is one of the significant VRAM consumption points in video/image generation pipelines.

The latent diffusion architecture uses a Variational Autoencoder (VAE) to compress images into a low-dimensional latent space, performing noise addition and denoising in the compressed space, which is the origin of the name Stable Diffusion.

Bayesian inference in probability theory is the theoretical foundation of generative models such as VAEs and diffusion models.

The core innovation of VAE lies in encoding inputs as a probability distribution (typically a Gaussian distribution described by mean μ and variance σ) rather than a deterministic point, making the latent space continuous, smooth, and sampleable.

扩散模型在生成图像时本质上是在高维潜空间（Latent Space）中进行去噪采样，缺乏多角度约束时同一角色在不同帧中的特征易产生语义漂移

Stable Diffusion moves the diffusion process from pixel space to latent space, compressing images into low-dimensional representations via a VAE encoder before denoising, thereby reducing computational overhead.

The main model (checkpoint) is a complete weight package containing UNet, CLIP and VAE that determines the base art style and capabilities; the text encoder converts prompts into conditioning vectors; LoRA is a small low-rank adaptation incremental weight; and VAE encodes and decodes between latent space and pixel space.

The Diffusion model (UNet) is responsible for gradually denoising in latent space to generate content; the text encoder converts prompts into semantic vectors; the VAE is responsible for encoding and decoding between pixel space and latent space, and the versions of all three must match each other.

Locally deploying MiniMax H3 requires downloading five model files: the main model h3f12t8, the all-purpose reference model h3aef2vnt8 (both choosing the int8 version), the Qianwen minimax-h3vfp4 text encoder, and two VAE decoders for video and audio.

Ask4NV's joint training improved cross-view correspondences in the VAE representation.

Ask4NV injects camera geometry information into both the VAE and ViT representations, so that geometric priors are explicitly considered at the encoding stage.

The base model (Checkpoint) determines the overall art style and semantic understanding capability and is usually the largest in size; the VAE is responsible for decoding latent space data into visible images, directly affecting the color and detail of the generated image, and if it does not match the base model the image tends to be gray or color-shifted; LoRA is a small fine-tuning weight used to layer on a specific character, style, or action, occupying little VRAM and switching flexibly.

In experiments, the VAE's latent variable z can serve as a degradation fingerprint, compressing high-dimensional sensor data into a low-dimensional degradation state representation that is more robust to noise than raw sensor values.

A variational autoencoder (VAE) contains an encoder and a decoder, samples latent variables through the reparameterization trick, and its loss function consists of a reconstruction error and a KL divergence regularization term.