Journal of Global Optimization

, Volume 34, Issue 3, pp 441–466 | Cite as

Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models

  • D. HuangEmail author
  • T. T. Allen
  • W. I. Notz
  • N. Zeng


This paper proposes a new method that extends the efficient global optimization to address stochastic black-box systems. The method is based on a kriging meta-model that provides a global prediction of the objective values and a measure of prediction uncertainty at every point. The criterion for the infill sample selection is an augmented expected improvement function with desirable properties for stochastic responses. The method is empirically compared with the revised simplex search, the simultaneous perturbation stochastic approximation, and the DIRECT methods using six test problems from the literature. An application case study on an inventory system is also documented. The results suggest that the proposed method has excellent consistency and efficiency in finding global optimal solutions, and is particularly useful for expensive systems.


Efficient global optimization expected improvement kriging stochastic black-box systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ackley, D.H. 1987A Connectionist Machine for Genetic Hill-climbingKluwer Academic PublishersBostonCrossRefGoogle Scholar
  2. 2.
    Andradóttir, S. 1998Simulation optimizationBanks, J. eds. Handbook on SimulationWileyNew YorkChapter 9Google Scholar
  3. 3.
    Angün, E., Kleijnen, J.P., Hertog, D.D. and Gürkan, G. (2002), Response surface methodology revised. In: Proceedings of the 2002 Winter Simulation Conference, pp. 377–383.Google Scholar
  4. 4.
    Barton, R. R. 1984Minimization algorithms for functions with random noiseAmerican Journal of Mathematical and Management Sciences4109138Google Scholar
  5. 5.
    Barton, R.R., Ivey, J.S. 1996Nelder–Mead simplex modifications for simulation optimizationManagement Science42954973CrossRefGoogle Scholar
  6. 6.
    Branin, F.H. 1972Widely convergent methods for finding multiple solutions of simultaneous nonlinear equationsIBM Journal of Research Developments16504522CrossRefGoogle Scholar
  7. 7.
    Branin, F.H., Hoo, S.K. 1972A method for finding multiple extrema of a function of n variablesLootsma, F.A. eds. Numerical Methods of Nonlinear OptimizationAcademic PressLondon231237Google Scholar
  8. 8.
    Cressie, N.A.C. 1993Statistics for Spatial DataWileyNew Yorkrevised editionGoogle Scholar
  9. 9.
    Currin, C., Mitchell, M., Morris, M., Ylvisaker, D. 1991Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experimentsJournal of American Statistics Association86953963CrossRefGoogle Scholar
  10. 10.
    Fang, K.-T., Lin, D.K., Winker, P., Zhang, Y. 2000Uniform design: theory and applicationTechnometrics42237248CrossRefGoogle Scholar
  11. 11.
    Fu, M.C. 1994Optimization via simulation: a reviewAnnals of Operations Research53199247CrossRefGoogle Scholar
  12. 12.
    Gablonsky, J.M., Kelley, C.T. 2001A locally-biased form of the direct algorithmJournal of Global Optimization212737CrossRefGoogle Scholar
  13. 13.
    Haftka, R.T., Gürdal, Z. 1993Elements of Structural Optimization3KluwerBoston, MAGoogle Scholar
  14. 14.
    Hartman, J.K. 1973Some experiments in global optimizationNaval Research Logistics Quarterly20569576CrossRefGoogle Scholar
  15. 15.
    Humphrey, D.G., Wilson, J.R. 2000A revised simplex search procedure for stochastic simulation response-surface optimizationINFORMS Journal on Computing12272283CrossRefGoogle Scholar
  16. 16.
    Johnson, M.E., Moore, L.M., Ylvisaker, D. 1990Minimax and maximin distance designsJournal of Statistical Planning and Inference26131148CrossRefGoogle Scholar
  17. 17.
    Jones, D., Schonlau, M., Welch, W. 1998Efficient global optimization of expensive black-box functionsJournal of Global Optimization13455492CrossRefGoogle Scholar
  18. 18.
    Kiefer, J., Wolfowitz, J. 1952Stochastic estimation of a regression functionAnnals of Mathematical Statistics23462466CrossRefGoogle Scholar
  19. 19.
    Koehler, J.R., Owen, A.B. 1996Computer experimentsGhosh, S.Rao, C.R. eds. Handbook of StatisticsElsevier Science B.V.Amsterdam, the NetherlandsVol. 13Google Scholar
  20. 20.
    Kushner, H.J. 1964A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noiseJournal of Basic Engineering8697106CrossRefGoogle Scholar
  21. 21.
    Kushner, H.J., Clark, D.C. 1978Stochastic Approximation Methods for Constrained and Unconstrained SystemsSpringerNew YorkGoogle Scholar
  22. 22.
    Law, A.M., Kelton, W.D. 2000Simulation Modeling and Analysis3McGraw-HillBoston6080Google Scholar
  23. 23.
    Lindley, D.V. 1956On a measure of information provided by an experimentAnnals of Mathematical Statistics279861005CrossRefGoogle Scholar
  24. 24.
    McKay, M.D., Beckman, R.J., Conover, W.J. 1979A comparison of three methods for selecting values of input variables in the analysis of output from a computer codeTechnometrics21239245CrossRefGoogle Scholar
  25. 25.
    Neddermeijer, H.G., Piersma, N., van Oortmarssen, G.J., Habbema, J.D.F. and Dekker, R. (1999), Comparison of response surface methodology and the Nelder and Mead simplex method for optimization in microsimulation models, Econometric Institute Report EI-9924/A, Erasmus University Rotterdam, The Netherlands.Google Scholar
  26. 26.
    Neddermeijer, H.G., van Oortmarssen, G.J., Piersma N. and Dekker R. (2000), A framework for response surface methodology for simulation optimizaton. In: Proceedings of the 2000 Winter Simulation Conference, pp. 129–136.Google Scholar
  27. 27.
    Niederreiter, H. 1992Random number generation and quasi-Monte-Carlo methodsSAIMPhiladelphiaCrossRefGoogle Scholar
  28. 28.
    O’Hagan, A. 1989Comment: design and analysis of computer experimentsStatistic Science4430432CrossRefGoogle Scholar
  29. 29.
    Perttunen, C.D., Jones, D.R., Stuckman, B.E. 1993Lipschitzian optimization without the lipschitz constantJournal of Optimization Theory and Application79157181CrossRefGoogle Scholar
  30. 30.
    Rodriguez, J.F., Perez, V.M., Padmanabhan, D., Renaud, J.E. 2001Sequential approximate optimization using variable fidelity response surface approximationStructure Multidiscipline Optimization222334(Springer-Verlag)Google Scholar
  31. 31.
    Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P. 1989aDesign and analysis of computer experiments (with discussion)Statistical Science4409435CrossRefGoogle Scholar
  32. 32.
    Sacks, J., Schiller, S.B., Welch, W. 1989bDesign for computer experimentsTechnometrics314147CrossRefGoogle Scholar
  33. 33.
    Safizadeh, M.H., Thornton, B.M. 1984Optimization in simulation experiments using response surface methodologyComparative Industrial Engineering81127CrossRefGoogle Scholar
  34. 34.
    Santner, T.J., Williams, B.J., Notz, W.I. 2003The Design and Analysis of Computer ExperimentsSpringerNew YorkGoogle Scholar
  35. 35.
    Sasena, M.J., Papalambros, P.Y. and Goovaerts, P. (2001), The use of surrogate modeling algorithms to exploit disparities in function computation time within simulation-based optimization. In: Proceedings of the 4th Congress on Structural and Multidisciplinary Optimization, Dalian, China, June 4–8, 2001.Google Scholar
  36. 36.
    Sasena, M.J., Papalambros, P.Y., Goovaerts, P. 2002Exploration of metamodeling sampling criteria for constrained global optimizationEngineering Optimization34263278CrossRefGoogle Scholar
  37. 37.
    Sasena, M.J. (2002), Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations, Ph.D. dissertation, University of Michigan.Google Scholar
  38. 38.
    Sóbester, A., Leary, S. and Keane, A.J. (2002), A parallel updating scheme for approximating and optimizing high fidelity computer simulation. In: The Third ISSMO/AIAA Internet Conference on Approximations in Optimization, October 14–18, 2002.Google Scholar
  39. 39.
    Spall, J.C. 1992Multivariate stochastic approximation using a simultaneous perturbation gradient approximationIEEE Transactions on Automatic Control37332341CrossRefGoogle Scholar
  40. 40.
    Spall, J.C. 1998Implementation of the simultaneous perturbation algorithm for stochastic optimizationIEEE Transactions on Aerospace and Electronics Systems34817823CrossRefGoogle Scholar
  41. 41.
    Stein, M. 1987Large sample properties of simulation using Latin hypercube samplingTechnometrics29143151CrossRefGoogle Scholar
  42. 42.
    Williams, B.J., Santner, T.J., Notz, W.I. 2000Sequential design of computer experiments to minimize integrated response functionsStatistica Sinica1011331152Google Scholar
  43. 43.
    Žilinskas, A. 1980MINUN – Optimization of one-dimensional multimodal functions in the presence of noise, algorithms 44Aplikace Matematiky25392402Google Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  1. 1.Scientific Forming Technologies CorporationColumbusUSA
  2. 2.Department of Industrial, Welding, and Systems EngineeringThe Ohio State UniversityColumbusUSA
  3. 3.Department of StatisticsThe Ohio State UniversityColumbusUSA

Personalised recommendations