Bayesian Optimization

Reinforcement learning learns optimal policies through continuous interaction between an agent and its environment; its mathematical foundation is the Markov Decision Process (MDP), which consists of a state space, action space, transition function, reward function, and discount factor.

In combinatorial spaces with dimensionality up to 6000+, traditional BO cannot run due to computational bottlenecks, whereas the new method can search efficiently.

A new paper proposes restructuring Bayesian optimization through linear structure, making it up to three orders of magnitude faster while maintaining optimization performance comparable to state-of-the-art methods.

The traditional view holds that Bayesian optimization is only suitable for expensive functions and that algorithm speed does not matter, but problem cost is actually a continuum, from cheap to expensive.

On the speed-performance Pareto frontier, existing Bayesian optimization methods sit far from the high-speed end due to their high computational cost, while evolutionary strategies such as CMA-ES are fast but have low sample efficiency, leaving a gap in the middle of the frontier.

The new method applies to both continuous and combinatorial inputs, with dimensionality ranging from about 10 to 6000+.

Experiments show that under the same evaluation budget, the new method's performance is comparable to state-of-the-art BO but it is 100 to 1000 times faster.

Common hyperparameter tuning strategies include grid search, random search, and Bayesian optimization; the latter models historical evaluation results to guide the next sampling and typically achieves better results with fewer trials.

AI (especially reinforcement learning and Bayesian optimization) has been used to automatically search for optimal pulse shapes and timings to maximize quantum gate fidelity, and can continuously adjust online when hardware characteristics drift over time.

The exploration strategy proposed in this study is: in any Bayesian optimization procedure, occasionally replace the acquisition query with an exploration query.

In practical deployment, Bayesian optimization typically cares only about the quality of the single recommended point that the algorithm finally returns, rather than the intermediate performance of the entire optimization process.

This study proposes that the existing cumulative regret guarantees of Bayesian optimization do not necessarily characterize the quality of the algorithm's final output.

This exploration query replacement scheme does not change the main structure of the original algorithm and can be seamlessly integrated into existing Bayesian optimization frameworks.

On synthetic benchmarks, a hyperparameter optimization task and a control task, after introducing the exploration strategy, the final recommendation quality of existing Bayesian optimization algorithms was improved in all cases.

Expected improvement (EI) is a classic acquisition function in Bayesian optimization that computes the expected amount of improvement possibly obtained relative to the incumbent after sampling at a candidate point, and it usually has a closed-form expression.

The computational complexity of Bayesian optimization typically grows exponentially with parameter dimensionality, and it is traditionally only applicable to low-dimensional problems (generally within a few dozen dimensions).

AI agents do not try to exhaustively enumerate all combinations, but instead use strategies such as heuristic search, machine learning predictive models or Bayesian optimization to quickly locate high-potential regions in a vast space, compressing an unmanageable search space to a scale that can be experimentally verified.

Standard Gaussian process Bayesian optimization is not suitable for spatio-temporal point process problems because its assumption of approximately stationary, independent observation noise is inconsistent with the data-generating mechanism of a doubly stochastic Cox process.

In reaction optimization tasks, RSI-Anything achieved an average best-observed yield of 96.15% under a budget of 40 experiments, outperforming Bayesian optimization's 82.16%

Bayesian optimization is a sequential experimental design method for expensive black-box functions that typically uses Gaussian processes as a probabilistic surrogate model, and has long been regarded as a strong baseline in the field of experimental design