A Warped Kernel Improving Robustness in Bayesian Optimization Via Random Embeddings
This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates constraints on bound selection for the embedded domain, thus improving the robustness, as illustrated with a test case with 25 variables and intrinsic dimension 6.
KeywordsBlack-box optimization Expected Improvement Low-intrinsic dimensionality Gaussian processes REMBO
This work has been conducted within the frame of the ReDice Consortium, gathering industrial (CEA, EDF, IFPEN, IRSN, Renault) and academic (Ecole des Mines de Saint-Etienne, INRIA, and the University of Bern) partners around advanced methods for Computer Experiments.
The authors also thanks the anonymous reviewers as well as Frank Hutter for their helpful suggestions.
- 4.Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L., Szego, G. (eds.) Towards Global Optimization, vol. 2, pp. 117–129. Elsevier, Amsterdam (1978)Google Scholar
- 6.Roustant, O., Ginsbourger, D., Deville, Y.: DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J. Stat. Softw. 51(1), 1–55 (2012)Google Scholar
- 7.Snoek, J., Swersky, K., Zemel, R.S., Adams, R.P.: Input warping for Bayesian optimization of non-stationary functions. In ICML (2014)Google Scholar
- 9.Wang, Z., Zoghi, M., Hutter, F., Matheson, D., de Freitas, N.: Bayesian optimization in high dimensions via random embeddings. In: IJCAI (2013)Google Scholar