Exploring or reducing noise?

A global optimization algorithm in the presence of noise
  • Didier RullièreEmail author
  • Alaeddine Faleh
  • Frédéric Planchet
  • Wassim Youssef
Research Paper


We consider the problem of the global minimization of a function observed with noise. This problem occurs for example when the objective function is estimated through stochastic simulations. We propose an original method for iteratively partitioning the search domain when this area is a finite union of simplexes. On each subdomain of the partition, we compute an indicator measuring if the subdomain is likely or not to contain a global minimizer. Next areas to be explored are chosen in accordance with this indicator. Confidence sets for minimizers are given. Numerical applications show empirical convergence results, and illustrate the compromise to be made between the global exploration of the search domain and the focalization around potential minimizers of the problem.


Global optimization Noise Potential Branch-and-Bound Simplex Kriging 



The authors would like to thank the anonymous reviewers and professor Ragnar Norberg for their valuable comments and suggestions.


  1. Aarts EHL, Laarhoven V (1985) Statistical cooling: a general approach to combinatorial optimization problems. Philips J Res 40(4):193–226MathSciNetGoogle Scholar
  2. Alliot JM (1996) Techniques d’optimisation stochastique appliquées aux problèmes du contrle aérien. INPT, Habilitation Diriger des RecherchesGoogle Scholar
  3. Arora JS, Elwakeil OA, Chahande AI, Hsieh CC (1995) Global optimization methods for engineering applications: a review. Struct Optim 9:137–159CrossRefGoogle Scholar
  4. Bect J (2010) IAGO for global optimisation with noisy evaluations. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22-24 Nov. 2010. Slides available at
  5. Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton, NJzbMATHGoogle Scholar
  6. Blum JR (1954) Multidimensional stochastic approximation methods. Ann Math Stat 25:737–744zbMATHCrossRefGoogle Scholar
  7. Box GEP, Draper NR (2007) Response surfaces, mixtures, and ridge analyses. WileyGoogle Scholar
  8. Branke J, Meisel S, Schmidt C (2008) Simulated annealing in the presence of noise. Journal of Heuristics 14(6):627–654CrossRefGoogle Scholar
  9. Broadie M, Cicek DM, Zeevi A (2009) An adaptative multidimensional version of the kiefer-Wolfowitz stochastic. In: Rossetti MD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceeding of the 2009 winter simulation conferenceGoogle Scholar
  10. De Berg M, Cheong O, van Kreveld M, Overmars M (2008) Computational geometry: algorithms and applications. Springer-VerlagGoogle Scholar
  11. Emmerich MTM (2005) Single and multi-objective evolutionary design optimization assisted by Gaussian random field Metamodels. Dissertation zur Erlangung des Grades eines Doktors der Naturwissenschaften der Universit¨at Dortmund, DortmundGoogle Scholar
  12. Garcia MJ, Gonzalez CA (2004) Shape optimisation of continuum structures via evolution strategies and fixed grid finite element analysis. Struct Multidisc Optim 26:92–98CrossRefGoogle Scholar
  13. Ginsbourger D (2009) Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables. Thèse de doctorat de mathématiques appliquées, Ecole nationale supéerieure des mines de Saint-Etienne, n519MAGoogle Scholar
  14. Gu X, Renaud JE, Batill SM, Brach RM, Budhiraja AS (2000) Worst case propagated uncertainty of multidisciplinary systems in robust design optimization. Struct Multidisc Optim 20:190–213CrossRefGoogle Scholar
  15. Horst R, Pardalos PM (1995) Handbook of global optimization. Kluwer Academic Publishers, Dordrecht Boston LondonzbMATHGoogle Scholar
  16. Hansen ER (1979) Global optimization using interval analysis: the one dimensional case. JOTA 29:331–344zbMATHCrossRefGoogle Scholar
  17. Janusevskis J, Le Riche R (2010) Simultaneous kriging-based sampling for optimization and uncertainty propagation. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov 2010Google Scholar
  18. Jones DR, Pertunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theory Appl 79(1):157–181MathSciNetzbMATHCrossRefGoogle Scholar
  19. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492MathSciNetzbMATHCrossRefGoogle Scholar
  20. Jones DR (2001) A taxonomy of global optimization methods based on response surface. J Glob Optim 21:345–383zbMATHCrossRefGoogle Scholar
  21. Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann Math Stat 23:462–466MathSciNetzbMATHCrossRefGoogle Scholar
  22. Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192:707–716MathSciNetzbMATHCrossRefGoogle Scholar
  23. Kleijnen JPC, van Beers W, Van Nieuwenhuyse I (2011) Expected improvement in efficient global optimization through bootstrapped kriging. Springer, pp 1–5Google Scholar
  24. Lawler EL, Wood DE (1966) Branch and Bound methods: a survey. Oper Res 14(4):699–719MathSciNetzbMATHCrossRefGoogle Scholar
  25. Lucasius CB, Kateman G (1993) Understanding and using genetic algorithms Part 1. Concepts, properties and context. Chemometr Intell Lab Syst 19(1):1–33CrossRefGoogle Scholar
  26. Mathias K, Whitley D, Kusuma A, Stork C (1996) An empirical evaluation of genetic algorithms on noisy objective functions. CRC, pp 65–86Google Scholar
  27. Nelder J, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313zbMATHCrossRefGoogle Scholar
  28. Norkin V, Pflug GCh, Ruszczyński A (1996) A branch and bound method for stochastic global optimization. Math Program 83(1–3):452–450Google Scholar
  29. Picheny V, Ginsourger D, Richet Y (2010) Optimization of noisy computer experiments with tunable precision. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov. 2010. Slides available at
  30. Robbins H, Monro S (1951) A Stochastic approximation method. Ann Math Stat 22:400–407MathSciNetzbMATHCrossRefGoogle Scholar
  31. Rullière D, Ribereau P (2011) Information aggregation and kriging alternative in a noisy environment. Preprint. French version available on HALGoogle Scholar
  32. Sakata S, Ashida F (2009) Ns-kriging based microstructural optimization applied to minimizing stochastic variation of homogenized elasticity of fiber reinforced composites. Struct Multidisc Optim 38:443–453CrossRefGoogle Scholar
  33. Schonlau M (1997) Computer experiments and global optimization. PhD, Dissertation, University of WaterlooGoogle Scholar
  34. Schubert B (1972) A sequential method seeking the global maximum of a function. SIAM J Numer Anal 9:379–388MathSciNetCrossRefGoogle Scholar
  35. Shin YS, Grandhi RV (2001) A global structural optimization technique using an interval method. Struct Multidisc Optim 22:351–363CrossRefGoogle Scholar
  36. Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27:302–313CrossRefGoogle Scholar
  37. Smith NA, Tromble RW (2004) Sampling uniformly from the unit simplex. Technical Report, Johns Hopkins UniversityGoogle Scholar
  38. Strugarek C (2006) Approches variationnelles et autres contributions en optimisation stochastique. ENPC, Thèse de doctoratGoogle Scholar
  39. Vazquez E, Villemonteix J, Sidorkiewicz M, Walter E (2008) Global optimization based on noisy evaluations: an empirical study of two statistical approaches. J Phys Conference Series 135: 012100Google Scholar
  40. Villemonteix J (2009) Optimisation de fonctions coûteuses. Thèse de doctorat de physique, Université Paris Sud 11, Faculté des sciences d’Orsay, n9278Google Scholar
  41. Woon SY, Querin OM, Steven GP (2001) Structural application of a shape optimization method based on a genetic algorithm. Struct Multidisc Optim 22:57–64CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Didier Rullière
    • 1
    Email author
  • Alaeddine Faleh
    • 2
  • Frédéric Planchet
    • 3
  • Wassim Youssef
    • 3
  1. 1.Ecole ISFA, Laboratoire SAFUniversité de Lyon, Université Lyon 1LyonFrance
  2. 2.Caisse des Dèpôts et ConsignationsParisFrance
  3. 3.Winter & AssociésParisFrance

Personalised recommendations