Abstract
This chapter discusses Bayesian Optimization for problems without constraints. A general algorithm is introduced, along with discussion of the importance of the choice of the acquisition function. Among a larger list of examples of acquisition functions, three such functions are examined in more detail: probability of improvement, expected improvement, and lower confidence bound. These three are demonstrated on both simple and more complex examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baker E, Barbillon P, Fadikar A, Gramacy RB, Herbei R, Higdon D, Huang J, Johnson LR, Ma P, Mondal A, Pires B, Sacks J, Sokolov V (2021) Analyzing stochastic computer models: a review with opportunities. Stat Sci (to appear)
Berger JO (1985) Statistical decision theory and Bayesian analysis, 2nd edn. Springer series in statistics. Springer, New York
Brochu E, Cora VM, de Freitas N (2010) A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Tech. Rep. 1012.2599, arXiv
Broyden CG (1970) The convergence of a class of double-rank minimization algorithms. IMA J Appl Math 6(1):76–90
Cox DD, John S (1997) Sdo: A statistical method for global optimization. In: Alexandrov NM, Hussaini MY (eds) Multidisciplinary Design Optimization: State of the Art. SIAM, Philadelphia, pp 315–329
Fletcher R (1970) A new approach to variable metric algorithms. Comput J 13(3):317–322
Goldfarb DF (1970) A family of variable-methods derived by variational means. Math Comput 24:23–26
Gramacy RB (2020) Surrogates: Gaussian process modeling, design, and optimization for the applied sciences. Chapman & Hall/CRC, Boca Raton
Gramacy RB, Lee HKH (2012) Cases for the nugget in modeling computer experiments. Stat Comput 22:713–722
Henning P, Schuler CJ (2012) Entropy search for information-efficient global optimization. J Mach Learn Res 13:1809–1837
Hernández-Lobato JM, Hoffman MW, Ghahramani Z (2014) Predictive entropy search for efficient global optimization of black-box functions. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger K (eds) Advances in neural information processing systems, vol 27. Curran Associates, Red Hook
Jalali H, Van Nieuwenhuyse I, Picheny V (2017) Comparison of kriging-based methods for simulation optimization with heterogeneous noise. Eur J Oper Res 261(1):279–301
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black box functions. J Global Optim 13:455–492
Kennedy M, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B (Stat Methodol) 63(2):425–464
Kushner HJ (1964) A new method of locating the maximum of an arbitrary multipeak curve in the presence of noise. J Basic Eng 86:97–106
Mockus J, Tiesis V, Zilinskas A (1978) The application of Bayesian methods for seeking the extrenum. Towards Global Optim 2:117–129
Nelder J, Mead R (1965) A simplex method for function optimization. Comput J 7(4):308–313
Picheny V, Ginsbourger D (2014) Noisy kriging-based optimization methods: a unified implementation within the diceoptim package. Comput Stat Data Anal 71:1035–1053
Picheny V, Wagner T, Ginsbourger D (2013) A benchmark of kriging-based infill criteria for noisy optimization. Struct Multidiscipl Optim 48(3):607–626
Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer, New York, NY
Shahriari B, Swersky K, Wang Z, Adams RP, de Freitas N (2016) Taking the human out of the loop: A review of Bayesian optimization. Proc IEEE 104(1):148–175
Shanno D (1970) Conditioning of quasi-Newton methods for function minimization. Math Comput 24:647–656
Srinivas N, Krause A, Kakade S, Seeger M (2010) Gaussian process optimization in the bandit setting: No regret and experimental design. In: Proceedings of the 27th international conference on machine learning, ICML 2010
Thompson WR (1933) On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3/4):285–294
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pourmohamad, T., Lee, H.K.H. (2021). Unconstrained Optimization. In: Bayesian Optimization with Application to Computer Experiments. SpringerBriefs in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-82458-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-82458-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82457-0
Online ISBN: 978-3-030-82458-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)