Diffusion Models Without Training Data: A New Breakthrough in Lattice Field Theory Sampling

In lattice field theory, sampling field configurations from equilibrium distributions has long been a core bottleneck in computational physics. Traditional methods such as Hybrid Monte Carlo (HMC) are mature, but the computational cost of generating independent configurations rises sharply with system size. Diffusion models have shown potential as generative samplers, yet their standard denoising score matching objective requires sampling from the target distribution—which brings us right back to the original problem. A recent study proposes a "data-free training" scheme that allows diffusion models to train solely on their own generated samples, achieving performance comparable to HMC-reference-data training in two-dimensional φ⁴ theory.

The Core Contradiction: Why Does Training a Diffusion Model Require "Answers"?

The learning process of diffusion models relies on denoising score matching. Put simply, the model must learn to recover original configurations from noisy ones, and this process requires that the training data itself come from the target distribution. In lattice field theory, the target distribution is precisely the equilibrium distribution we want to sample. This creates a logical loop: to train a sampler, you first need to be able to sample.

The traditional solution is to use HMC to generate a reference dataset, train a diffusion model on this data, and then use the trained model to rapidly generate large numbers of independent configurations. While effective, this approach does not truly bypass the computational overhead of HMC—it merely shifts the cost to the training phase. For critical regions or large-volume systems, HMC's autocorrelation time can be extremely long, and the cost of generating sufficient training data remains high.

Denoising Score Matching (DSM) is the core objective function for training diffusion models. The basic idea is: first apply Gaussian noise of varying strengths to real data, then train a network to predict the gradient direction of the added noise—this gradient is exactly proportional to the score function of the perturbed distribution. Mathematically, DSM transforms the intractable score matching loss into a tractable regression problem, where the regression targets come directly from noise-corrupted real data points. Therefore, the training set must consist of samples from the target distribution; otherwise, the network learns the score of the wrong distribution. This dependency is precisely the fundamental obstacle when applying diffusion models to lattice field theory: the target distribution is itself an unknown equilibrium distribution, and there is no cheap sampling method other than HMC. This structure of DSM also explains why early work had to first use HMC to generate a training set before training a diffusion model, forming a two-stage "sample first, accelerate later" pipeline.

Hybrid Monte Carlo (HMC) is the standard algorithm for generating equilibrium configurations in lattice field theory. It evolves field configurations using Hamiltonian dynamics and ensures sampling correctness through a Metropolis accept-reject step. HMC's notable advantage is its ability to exploit gradient information of the action, making it far more efficient than traditional Metropolis algorithms in continuum field theory. However, HMC is still a Markov chain Monte Carlo method, and adjacent configurations exhibit autocorrelation: obtaining a statistically independent sample requires a sufficient number of evolution steps. Near critical points, the autocorrelation time diverges with system volume (critical slowing down), causing HMC's sampling cost to rise sharply. This "autocorrelation bottleneck" is precisely the target that generative methods like diffusion models aim to break through, and it is also the application scenario where data-free training schemes hold the greatest value.

Target Score Identity: Replacing Training Data with Unnormalized Density

The study's entry point is a mathematical relationship called the "target score identity." The score function that a diffusion model needs to learn is the gradient of the probability density with respect to the configuration. The key insight is: if one can obtain the unnormalized probability density (in lattice field theory, the action is known, so this condition is naturally satisfied), then an identity can be constructed that expresses the score function as a self-consistent relationship between the current model prediction and the density gradient.

This means the model no longer needs external data to define the training objective. In each training round, the model starts from its own generated samples, uses the target score identity to compute the loss, and updates parameters. The paper explores two implementation paths: an importance-weighted version that provides a consistent objective throughout training, and an unweighted variant that defines a self-consistent fixed-point training scheme. Each has trade-offs—the former is theoretically more stable, while the latter is simpler to implement.

Score function in generative models refers to the log-gradient of the probability density with respect to configuration variables, i.e., ∇_x log p(x). It does not depend on the normalization constant of the density, so as long as the unnormalized probability density is known (in lattice field theory, exp(−S[φ]), where S is the Euclidean action), the score function can in principle be obtained. The reverse generation process of diffusion models learns this score function to gradually restore noise into samples from the target distribution. Precisely because the score function is insensitive to the normalization constant, it is naturally suited to lattice field theory—computing the partition function is itself a difficult problem, and the score function bypasses it. The target score identity further exploits this property by expressing the score function as a self-consistent relationship between model predictions and density gradients, thereby enabling the construction of training objectives without sampling from the target distribution. This identity is the mathematical foundation that makes data-free training viable, and its form has intrinsic connections to methods such as force matching and score matching in statistical physics.

Importance weighting in machine learning is used to correct for bias between the training distribution and the target distribution. In the importance-weighted version of data-free training, the model's own generated samples come from a distribution that deviates from the target distribution. By assigning appropriate weights to each sample, the weighted loss function can be made equivalent in expectation to the loss on the target distribution, thereby providing a consistent objective. The cost of this consistency is that the weights may have large variance, especially when the model-generated distribution differs significantly from the target distribution, causing weights to become extreme and training to become unstable. The unweighted variant abandons the consistency guarantee and instead defines a self-consistent fixed-point condition: at the optimal solution, the model-generated distribution coincides with the target distribution, and the loss function automatically reaches its minimum, so iterative updates can gradually approach the fixed point. The trade-off between these two paths reflects the classic tension between stability and simplicity in generative model training.

Two-Dimensional φ⁴ Theory Benchmark: Performance in Symmetric, Critical, and Broken Phases

The research team conducted systematic benchmarking on two-dimensional φ⁴ scalar field theory, covering three typical physical regions: the symmetric phase, the critical phase, and the broken phase. These three regions pose different challenges to samplers: in the symmetric phase, field configuration fluctuations are small and sampling is relatively easy; the critical phase exhibits long-range correlations and long autocorrelation times, making it difficult for traditional methods; the broken phase involves vacuum selection and non-perturbative effects.

Experimental results show that a properly designed data-free training scheme can achieve efficient training and sampling in all three regions. More importantly, its sampling quality is comparable to that of diffusion models trained with HMC reference data. This means that without any prior sampling data, diffusion models are already capable of handling sampling tasks in lattice field theory.

Practical Significance and Limitations of Data-Free Training

From a methodological perspective, this work pushes the role of diffusion models in scientific computing from "accelerator" to "bootstrapper." The traditional approach uses diffusion models to replace HMC's sampling step, but training still depends on HMC data; the new scheme completely severs this dependency, making diffusion models truly independent sampling tools. This property is especially valuable for regions that HMC struggles to handle efficiently—such as large-volume systems near critical points.

However, current validation remains limited to two-dimensional φ⁴ theory. When extending to four-dimensional gauge field theory, although obtaining the unnormalized density is formally feasible (the action is known), the parameterization complexity of the score function, training stability, and mode collapse risks may all increase significantly. Additionally, the computational overhead and variance control of the importance-weighted version are factors that need to be weighed in practical deployment. The authors also note that the choice of variant needs to be adjusted according to the specific physical region, and a general recipe has yet to emerge.

Potential Impact on the Computational Paradigm of Lattice Field Theory

If data-free training can maintain its effectiveness in higher dimensions and more complex gauge theories, it will change the computational workflow of lattice field theory. Researchers will no longer need to run expensive HMC separately for each parameter point to generate training sets; instead, they can directly train a sampler and use it to generate independent configurations needed for physical observables. This could bring significant efficiency gains for studies requiring scanning over large parameter spaces, such as phase diagram mapping and continuum limit extrapolation.

Of course, there is still a long way to go from two-dimensional scalar field theory to four-dimensional quantum chromodynamics. But this study at least proves one thing: diffusion models can learn to sample from a target distribution without relying on samples from that distribution—as long as we know the functional form of the probability density. This idea may also inspire other scientific computing problems that require sampling from known unnormalized densities.

One-line Summary

Research proposes a data-free training scheme that enables diffusion models to learn to sample from lattice field theory equilibrium distributions using only the unnormalized density.

Paragraph Summary

Sampling from equilibrium distributions in lattice field theory is a core bottleneck, and traditional HMC methods become prohibitively expensive as system size grows. Although diffusion models are generative samplers, their standard denoising score matching training requires samples from the target distribution—precisely the sampling problem one wants to solve. A recent study proposes a "data-free training" scheme based on the target score identity, using the unnormalized probability density (with known action) to construct training objectives without any prior sampling data. Through importance-weighted and unweighted implementation paths, data-free training achieves performance comparable to HMC-reference-data training in the symmetric, critical, and broken phases of two-dimensional φ⁴ theory. This method pushes diffusion models from "accelerator" to "bootstrapper," but is currently limited to two-dimensional scalar field theory, with challenges such as parameterization complexity and training stability remaining for extension to four-dimensional gauge theory.