Skip to main content

An informational approach to the global optimization of expensive-to-evaluate functions

Abstract

In many global optimization problems motivated by engineering applications, the number of function evaluations is severely limited by time or cost. To ensure that each evaluation contributes to the localization of good candidates for the role of global minimizer, a sequential choice of evaluation points is usually carried out. In particular, when Kriging is used to interpolate past evaluations, the uncertainty associated with the lack of information on the function can be expressed and used to compute a number of criteria accounting for the interest of an additional evaluation at any given point. This paper introduces minimizers entropy as a new Kriging-based criterion for the sequential choice of points at which the function should be evaluated. Based on stepwise uncertainty reduction, it accounts for the informational gain on the minimizer expected from a new evaluation. The criterion is approximated using conditional simulations of the Gaussian process model behind Kriging, and then inserted into an algorithm similar in spirit to the Efficient Global Optimization (EGO) algorithm. An empirical comparison is carried out between our criterion and expected improvement, one of the reference criteria in the literature. Experimental results indicate major evaluation savings over EGO. Finally, the method, which we call IAGO (for Informational Approach to Global Optimization), is extended to robust optimization problems, where both the factors to be tuned and the function evaluations are corrupted by noise.

This is a preview of subscription content, access via your institution.

References

  1. 1

    Abrahamsen, P.: A review of Gaussian random fields and correlation functions. Tech. Rep., Norwegian Computing Center (1997). www.math.ntnu.no/~omre/TMA4250/V2007/abrahamsen2.ps

  2. 2

    Adler R.: On excursion sets, tubes formulas and maxima of random fields. Ann. Appl. Prob. 10(1), 1–74 (2000)

    Google Scholar 

  3. 3

    Chib S., Greenberg E.: Understanding the metropolis-hastings algorithm. Am. Stat. 49(4), 327–335 (1995)

    Article  Google Scholar 

  4. 4

    Chilès J., Delfiner P.: Geostatistics, Modeling Spatial Uncertainty. Wiley, New York (1999)

    Google Scholar 

  5. 5

    Cover T.M., Thomas A.J.: Elements of Information Theory. Wiley, New York (1991)

    Book  Google Scholar 

  6. 6

    Cox, D., John, S.: Sdo: a statistical method for global optimization. In: Alexandrov, N., Hussaini, M.Y. (eds.) Multidisciplinary Design Optimization: State of the Art, pp. 315–329. SIAM, Philadelphia (1997). citeseer.ifi.unizh.ch/cox97sdo.html

  7. 7

    Delfiner, P.: Shift invariance under linear models. Ph.D. thesis, Princetown University, New Jersey (1977)

  8. 8

    Dixon, L., Szegö, G.: The global optimisation problem: an introduction. In: Dixon, L., Szegö, G. (eds.) Towards Global Optimization 2. North-Holland Publishing Company (1978)

  9. 9

    Geman, D., Jedynak, B.: An active testing model for tracking roads in satellite images. Tech. Rep. 2757, Institut National de Recherche en Informatique et en Automatique (INRIA) (1995)

  10. 10

    Gutmann H.: A radial basis function method for global optimization. J. Glob. Optim. 19(3), 201–227 (2001)

    Article  Google Scholar 

  11. 11

    Huang, D.: Experimental planning and sequential Kriging optimization using variable fidelity data. Ph.D. thesis, Ohio State University (2005)

  12. 12

    Huang D., Allen T., Notz W., Zeng N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. J. Glob. Optim. 34, 441–466 (2006)

    Article  Google Scholar 

  13. 13

    Jones D.: A taxonomy of global optimization methods based on response surfaces. J.Glob. Optim. 21, 345–383 (2001)

    Article  Google Scholar 

  14. 14

    Jones D., Schonlau M., William J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)

    Article  Google Scholar 

  15. 15

    Matheron G.: Principles of geostatistics. Econ. Geol. 58, 1246–1266 (1963)

    Article  Google Scholar 

  16. 16

    Matheron, G.: Le krigeage universel. In: Cahiers du Centre de Morphologie Mathématique de Fontainebleau. Ecole des Mines de Paris (1969) Fasc.1

  17. 17

    Sasena M., Papalambros P., Goovaerts P.: Exploration of metamodeling sampling criteria for constrained global optimization. Eng. Opt. 34, 263–278 (2002)

    Article  Google Scholar 

  18. 18

    Schonlau, M.: Computer experiments and global optimization. Ph.D. thesis, University of Waterloo (1997)

  19. 19

    Sjö, E.: Crossings and maxima in Gaussian fields and seas. Ph.D. thesis, Lund Institute of Technology (2000)

  20. 20

    Smola, A.: Learning with kernels. Ph.D. thesis, Technische Universität Berlin (1998)

  21. 21

    Stein M.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York (1999)

    Google Scholar 

  22. 22

    Vechia A.: Estimation and model identification for continuous spatial processes. J. R. Stat. Soc. B(50), 297–312 (1998)

    Google Scholar 

  23. 23

    Wahba G.: Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. In: Schölkopf, B., Burges, C., Smola, A. (eds) Advances in Kernel Methods—Support Vector Learning, pp. 69–87. MIT Press, Boston (1998)

    Google Scholar 

  24. 24

    Watson A., Barnes R.: Infill sampling criteria to locate extremes. Math. Geol. 27(5), 589–698 (1995)

    Article  Google Scholar 

  25. 25

    Williams, C., Rasmussen, C.: Gaussian processes for regression. In: Touretzky, D., Mayer, M., Hasselmo, M. (eds.) Advances in Neural Information Processing Systems, vol. 8. MIT Press (1996)

  26. 26

    Williams B., Santner T., Notz W.: Sequential design of computer experiments to minimize integrated response functions. Stat. Sinica 10, 1133–1152 (2000)

    Google Scholar 

  27. 27

    Yaglom A.: Correlation Theory of Stationary and Related Random Functions I: Basic Results. Vol. 6, Springer Series in Statistics. Springer-Verlag, New-York (1986)

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Emmanuel Vazquez.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Villemonteix, J., Vazquez, E. & Walter, E. An informational approach to the global optimization of expensive-to-evaluate functions. J Glob Optim 44, 509 (2009). https://doi.org/10.1007/s10898-008-9354-2

Download citation

Keywords

  • Gaussian process
  • Global optimization
  • Kriging
  • Robust optimization
  • Stepwise uncertainty reduction