Abstract
The optimization of expensive-to-evaluate functions generally relies on metamodel-based exploration strategies. Many deterministic global optimization algorithms used in the field of computer experiments are based on Kriging (Gaussian process regression). Starting with a spatial predictor including a measure of uncertainty, they proceed by iteratively choosing the point maximizing a criterion which is a compromise between predicted performance and uncertainty. Distributing the evaluation of such numerically expensive objective functions on many processors is an appealing idea. Here we investigate a multi-points optimization criterion, the multipoints expected improvement (\(q-{\mathbb E}I\)), aimed at choosing several points at the same time. An analytical expression of the \(q-{\mathbb E}I\) is given when q = 2, and a consistent statistical estimate is given for the general case. We then propose two classes of heuristic strategies meant to approximately optimize the \(q-{\mathbb E}I\), and apply them to the classical Branin-Hoo test-case function. It is finally demonstrated within the covered example that the latter strategies perform as good as the best Latin Hypercubes and Uniform Designs ever found by simulation (2000 designs drawn at random for every q ∈ [1,10]).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Abrahamsen, P.: A review of gaussian random fields and correlation functions, 2nd edn. Tech. Rep. 917, Norwegian Computing Center, Olso (1997)
Antoniadis, A., Berruyer, J., Carmona, R.: Régression non linéaire et applications. Economica, Paris (1992)
Baker, C., Watson, L.T., Grossman, B., Mason, W.H., Haftka, R.T.: Parallel global aircraft configuration design space exploration. Practical parallel computing, 79–96 (2001)
Bishop, C.: Neural Networks for Pattern Recognition. Oxford Univ. Press, Oxford (1995)
Blum, C.: Ant colony optimization: introduction and recent trends. Physics of Life Review 2, 353–373 (2005)
development Core Team R: R: A language and environment for statistical computing (2006), http://www.R-project.org
Cressie, N.: Statistics for spatial data. Wiley series in probability and mathematical statistics (1993)
Dreyfus, G., Martinez, J.M.: Réseaux de neurones. Eyrolles (2002)
Eiben, A., Smith, J.: Introduction to Evolutionary Computing. Springer, Heidelberg (2003)
Emmerich, M., Giannakoglou, K., Naujoks, B.: Single-and multiobjective optimization assisted by gaussian random field metamodels. IEEE Transactions on Evolutionary Computation 10(4), 421–439 (2006)
Geman, D., Jedynak, B.: An active testing model for tracking roads in satellite images. Tech. rep., Institut National de Recherches en Informatique et Automatique (INRIA) (December 1995)
Genton, M.: Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research 2, 299–312 (2001)
Ginsbourger, D.: Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables. PhD thesis, Ecole Nationale Supérieure des Mines de Saint-Etienne (2009)
Ginsbourger, D., Le Riche, R., Carraro, L.: A multipoints criterion for parallel global optimization of deterministic computer experiments. In: Non-Convex Programming 2007 (2007)
Goria, S.: Evaluation d’un projet minier: approche bayésienne et options réelles. PhD thesis, Ecole des Mines de Paris (2004)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
Henkenjohann, N., Göbel, R., Kleiner, M., Kunert, J.: An adaptive sequential procedure for efficient optimization of the sheet metal spinning process. Qual. Reliab. Engng. Int. 21, 439–455 (2005)
Huang, D., Allen, T., Notz, W., Miller, R.: Sequential Kriging optimization using multiple fidelity evaluations. Sructural and Multidisciplinary Optimization 32, 369–382 (2006)
Huang, D., Allen, T., Notz, W., Zheng, N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. Journal of Global Optimization 34, 441–466 (2006)
Jones, D.: A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization 21, 345–383 (2001)
Jones, D., Pertunen, C., Stuckman, B.: Lipschitzian optimization without the lipschitz constant. Journal of Optimization Theory and Application 79(1) (1993)
Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 455–492 (1998)
Journel, A.: Fundamentals of geostatistics in five lessons. Tech. rep., Stanford Center for Reservoir Forecasting (1988)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE Intl. Conf. on Neural Networks, vol. 4, pp. 1942–1948 (1995)
Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)
Knowles, J.: Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionnary Computation (2005)
Koehler, J., Owen, A.: Computer experiments. Tech. rep., Department of Statistics, Stanford University (1996)
Kracker, H.: Methoden zur analyse von computerexperimenten mit anwendung auf die hochdruckblechumformung. Master’s thesis, Dortmund University (2006)
Krige, D.: A statistical approach to some basic mine valuation problems on the witwatersrand. J. of the Chem., Metal. and Mining Soc. of South Africa 52(6), 119–139 (1951)
Martin, J., Simpson, T.: A monte carlo simulation of the Kriging model. In: 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, AIAA, AIAA-2004-4483, August 30 - September 2 (2004)
Martin, J., Simpson, T.: Use of Kriging models to approximate deterministic computer models. AIAA Journal 43(4), 853–863 (2005)
Matheron, G.: Principles of geostatistics. Economic Geology 58, 1246–1266 (1963)
Matheron, G.: La théorie des variables régionalisées et ses applications. Tech. rep., Centre de Morphologie Mathématique de Fontainebleau, Ecole Nationale Supérieure des Mines de Paris (1970)
O’Hagan, A.: Bayesian analysis of computer code outputs: a tutorial. Reliability Engineering and System Safety 91(91), 1290–1300 (2006)
Paciorek, C.: Nonstationary gaussian processes for regression and spatial modelling. PhD thesis, Carnegie Mellon University (2003)
Ponweiser, W., Wagner, T., Biermann, D., Vincze, M.: Multiobjective optimization on a limited budget of evaluations using model-assisted S-metric selection. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 784–794. Springer, Heidelberg (2008)
Praveen, C., Duvigneau, R.: Radial basis functions and Kriging metamodels for aerodynamic optimization. Tech. rep., INRIA (2007)
Queipo, N., Verde, A., Pintos, S., Haftka, R.: Assessing the value of another cycle in surrogate-based optimization. In: 11th Multidisciplinary Analysis and Optimization Conference, AIAA (2006)
Rasmussen, C., Williams, K.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Ripley, B.: Stochastic Simulation. John Wiley and Sons, New York (1987)
Roustant, O., Ginsbourger, D., Deville, Y.: The DiceKriging package: Kriging-based metamodeling and optimization for computer experiments. In: The UseR! Conference, Agrocampus-Ouest, Rennes, France (2009)
Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Statistical Science 4(4), 409–435 (1989)
Santner, T., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Heidelberg (2003)
Sasena, M.: Flexibility and efficiency enhancements for constrained global design optimization with Kriging approximations. PhD thesis, University of Michigan (2002)
Sasena, M.J., Papalambros, P., Goovaerts, P.: Exploration of metamodeling sampling criteria for constrained global optimization. Journal of Engineering Optimization (2002)
Sasena, M.J., Papalambros, P.Y., Goovaerts, P.: Global optimization of problems with disconnected feasible regions via surrogate modeling. In: Proceedings of the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Atlanta, GA (2002)
Schonlau, M.: Computer experiments and global optimization. PhD thesis, University of Waterloo (1997)
Schonlau, M., Welch, W., Jones, D.: A data-analytic approach to bayesian global optimization. In: Proceedings of the A.S.A. (1997)
Ulmer, H., Streichert, F., Zell, A.: Evolution strategies assisted by gaussian processes with improved pre-selection criterion. Tech. rep., Center for Bioinformatics Tuebingen, ZBIT (2003)
Villemonteix, J.: Optimisation de fonctions coûteuses: Modèles gaussiens pour une utilisation théorie et pratique industrielle. PhD thesis, Université Paris-sud XI, Faculté des Sciences d’Orsay (2008)
Villemonteix, J., Vazquez, E., Walter, E.: An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization 44(4), 509–534 (2009)
Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)
Williams, C., Rasmussen, C.: Gaussian processes for regression. In: Advances in Neural Information Processing Systems, vol. 8 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ginsbourger, D., Le Riche, R., Carraro, L. (2010). Kriging Is Well-Suited to Parallelize Optimization. In: Tenne, Y., Goh, CK. (eds) Computational Intelligence in Expensive Optimization Problems. Adaptation Learning and Optimization, vol 2. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10701-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-10701-6_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10700-9
Online ISBN: 978-3-642-10701-6
eBook Packages: EngineeringEngineering (R0)