Abstract
We consider the 1D Expected Improvement optimization based on Gaussian processes having spectral densities converging to zero faster than exponentially. We give examples of problems where the optimization trajectory is not dense in the design space. In particular, we prove that for Gaussian kernels there exist smooth objective functions for which the optimization does not converge on the optimum.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bull A.D.: Convergence rates of efficient global optimization algorithms. J. Mach. Learn. Res. 12, 2879–2904 (2011)
Forrester A.I.J., Sóbester A, Keane A.J.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, London (2008)
Gutmann H.-M.: A radial basis function method for global optimization. J. Glob. Optim. 19, 201–227 (2001)
Jones D.R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21, 345–383 (2001)
Jones, D.R., Schonlau, M., Welch, W.J.: A data analytic approach to Bayesian global optimization. In: Proceedings of the ASA, Section on Physical and Engineering Sciences, pp. 186–191 (1997)
Jones D.R., Schonlau M., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)
Katznelson Y.: An Introduction to Harmonic Analysis. Cambridge University Press, Cambridge (2004)
Kincaid D., Cheney E.: Numerical Analysis: Mathematics of Scientific Computing. American Mathematical Society, Providence, RI (2002)
Locatelli M.: Bayesian algorithms for one-dimensional global optimization. J. Glob. Optim. 10, 57–76 (1997)
Mockus J.: Bayesian Approach to Global Optimization: Theory and Applications. Mathematics and its Applications: Soviet Series. Kluwer Academic, Dordrecht (1989)
Mpmath library: http://code.google.com/p/mpmath/
Mockus J., Tiesis V., Zilinskas A.: The application of bayesian methods for seeking the extremum. Towards Glob. Optim. 2, 117–129 (1978)
Rasmussen C.E., Williams C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge, MA (2006)
Schonlau, M.: Computer Experiments and Global Optimization. Ph.D. thesis, Waterloo, ON, Canada (1997)
Schonlau, M., Welch, W.J.: Global optimization with nonparametric function fitting. In: Proceedings of the ASA, Section on Physical and Engineering Sciences, pp. 183–186 (1996)
Torn A., Zilinskas A.: Global optimization. Springer, New York (1989)
Vazquez, E., Bect, J.: Pointwise consistency of the kriging predictor with known mean and covariance functions. In: mODa 9—Advances in Model-oriented Design and Analysis. Bertinoro, Italy, 14–19 June 2010
Vazquez E., Bect J.: Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Stat. Plan. Inference 140(11), 3088–3095 (2010)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yarotsky, D. Examples of inconsistency in optimization by expected improvement. J Glob Optim 56, 1773–1790 (2013). https://doi.org/10.1007/s10898-012-9936-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-012-9936-x