Gaussian Process

GP-UCB uses a Gaussian process prior to model the reward function and defines the correlation between adjacent configurations through a kernel function, thereby sharing information across the configuration space and reducing exploration cycles.

Standard Gaussian process inference requires computing the inverse of the kernel matrix or solving a linear system, with a time complexity of O(n^3), where n is the number of observation points.

The core of the new method lies in exploiting linear structure in existing BO algorithms: under certain conditions the Gaussian process surrogate model can be transformed into a linear model, reducing computational complexity from cubic to linear or quadratic.

A Gaussian process is a Bayesian nonparametric model that treats the unknown function as a stochastic process, fully characterized by a mean function and a kernel function.

In Gaussian process modeling, the kernel function encodes prior knowledge such as the smoothness and scale of the objective function.

A randomly initialized neural network of width m converges to a Gaussian process as m tends to infinity.

The joint covariance matrix for n observation points in a full Gaussian process is of size n×n, with a direct computation time complexity of O(n³) and memory footprint of O(n²).

When the likelihood function is non-Gaussian (e.g., count data, binary data, or heavy-tailed distributions), the posterior distribution of the latent field becomes analytically intractable even when the prior is a Gaussian process.

The study proves that jointly conditioning a Gaussian process prior on both linear equality observations and finite-dimensional inequality events is equivalent to sequential conditioning in the sense of Borel measures on the solution space.

Existing GP solvers can only condition on equality information (PDE residuals, boundary data, observational measurements) and cannot directly enforce inequality constraints.

Gaussian process (GP) solvers return a posterior distribution and can quantify solution uncertainty, rather than merely providing a point estimate as traditional numerical methods do.

Bayesian optimization is a sequential experimental design method for expensive black-box functions that typically uses Gaussian processes as a probabilistic surrogate model, and has long been regarded as a strong baseline in the field of experimental design

Compared to prompt-based guidance, fine-tuning produces more stable results and uses fewer tokens, but requires high-quality sample data, computational resources, and technical expertise.