Advertisement

A Warped Kernel Improving Robustness in Bayesian Optimization Via Random Embeddings

  • Mickaël Binois
  • David Ginsbourger
  • Olivier Roustant
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8994)

Abstract

This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates constraints on bound selection for the embedded domain, thus improving the robustness, as illustrated with a test case with 25 variables and intrinsic dimension 6.

Keywords

Black-box optimization Expected Improvement Low-intrinsic dimensionality Gaussian processes REMBO 

Notes

Acknowledgments

This work has been conducted within the frame of the ReDice Consortium, gathering industrial (CEA, EDF, IFPEN, IRSN, Renault) and academic (Ecole des Mines de Saint-Etienne, INRIA, and the University of Bern) partners around advanced methods for Computer Experiments.

The authors also thanks the anonymous reviewers as well as Frank Hutter for their helpful suggestions.

References

  1. 1.
    Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Koziel, S., Ciaurri, D.E., Leifsson, L.: Surrogate-based methods. In: Koziel, S., Yang, X.-S. (eds.) Comput. Optimization, Methods and Algorithms. SCI, vol. 356, pp. 33–59. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  3. 3.
    Matheron, G.: Principles of geostatistics. Econ. Geol. 58(8), 1246–1266 (1963)CrossRefGoogle Scholar
  4. 4.
    Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L., Szego, G. (eds.) Towards Global Optimization, vol. 2, pp. 117–129. Elsevier, Amsterdam (1978)Google Scholar
  5. 5.
    Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006). http://www.gaussianprocess.org/gpml/ MATHGoogle Scholar
  6. 6.
    Roustant, O., Ginsbourger, D., Deville, Y.: DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J. Stat. Softw. 51(1), 1–55 (2012)Google Scholar
  7. 7.
    Snoek, J., Swersky, K., Zemel, R.S., Adams, R.P.: Input warping for Bayesian optimization of non-stationary functions. In ICML (2014)Google Scholar
  8. 8.
    Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York (1999)CrossRefMATHGoogle Scholar
  9. 9.
    Wang, Z., Zoghi, M., Hutter, F., Matheson, D., de Freitas, N.: Bayesian optimization in high dimensions via random embeddings. In: IJCAI (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mickaël Binois
    • 1
    • 2
  • David Ginsbourger
    • 3
  • Olivier Roustant
    • 1
  1. 1.Mines Saint-Étienne, UMR CNRS 6158, LIMOSSaint-ÉtienneFrance
  2. 2.Renault S.A.S.GuyancourtFrance
  3. 3.Department of Mathematics and StatisticsUniversity of BernBernSwitzerland

Personalised recommendations