Skip to main content

Exploring or reducing noise?

A global optimization algorithm in the presence of noise

Abstract

We consider the problem of the global minimization of a function observed with noise. This problem occurs for example when the objective function is estimated through stochastic simulations. We propose an original method for iteratively partitioning the search domain when this area is a finite union of simplexes. On each subdomain of the partition, we compute an indicator measuring if the subdomain is likely or not to contain a global minimizer. Next areas to be explored are chosen in accordance with this indicator. Confidence sets for minimizers are given. Numerical applications show empirical convergence results, and illustrate the compromise to be made between the global exploration of the search domain and the focalization around potential minimizers of the problem.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  • Aarts EHL, Laarhoven V (1985) Statistical cooling: a general approach to combinatorial optimization problems. Philips J Res 40(4):193–226

    MathSciNet  Google Scholar 

  • Alliot JM (1996) Techniques d’optimisation stochastique appliquées aux problèmes du contrle aérien. INPT, Habilitation Diriger des Recherches

    Google Scholar 

  • Arora JS, Elwakeil OA, Chahande AI, Hsieh CC (1995) Global optimization methods for engineering applications: a review. Struct Optim 9:137–159

    Article  Google Scholar 

  • Bect J (2010) IAGO for global optimisation with noisy evaluations. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22-24 Nov. 2010. Slides available at http://www.imsv.unibe.ch/content/continuingeducation/nko_workshop/program/index_ger.html

  • Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton, NJ

    MATH  Google Scholar 

  • Blum JR (1954) Multidimensional stochastic approximation methods. Ann Math Stat 25:737–744

    MATH  Article  Google Scholar 

  • Box GEP, Draper NR (2007) Response surfaces, mixtures, and ridge analyses. Wiley

  • Branke J, Meisel S, Schmidt C (2008) Simulated annealing in the presence of noise. Journal of Heuristics 14(6):627–654

    Article  Google Scholar 

  • Broadie M, Cicek DM, Zeevi A (2009) An adaptative multidimensional version of the kiefer-Wolfowitz stochastic. In: Rossetti MD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceeding of the 2009 winter simulation conference

  • De Berg M, Cheong O, van Kreveld M, Overmars M (2008) Computational geometry: algorithms and applications. Springer-Verlag

  • Emmerich MTM (2005) Single and multi-objective evolutionary design optimization assisted by Gaussian random field Metamodels. Dissertation zur Erlangung des Grades eines Doktors der Naturwissenschaften der Universit¨at Dortmund, Dortmund

  • Garcia MJ, Gonzalez CA (2004) Shape optimisation of continuum structures via evolution strategies and fixed grid finite element analysis. Struct Multidisc Optim 26:92–98

    Article  Google Scholar 

  • Ginsbourger D (2009) Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables. Thèse de doctorat de mathématiques appliquées, Ecole nationale supéerieure des mines de Saint-Etienne, n519MA

  • Gu X, Renaud JE, Batill SM, Brach RM, Budhiraja AS (2000) Worst case propagated uncertainty of multidisciplinary systems in robust design optimization. Struct Multidisc Optim 20:190–213

    Article  Google Scholar 

  • Horst R, Pardalos PM (1995) Handbook of global optimization. Kluwer Academic Publishers, Dordrecht Boston London

    MATH  Google Scholar 

  • Hansen ER (1979) Global optimization using interval analysis: the one dimensional case. JOTA 29:331–344

    MATH  Article  Google Scholar 

  • Janusevskis J, Le Riche R (2010) Simultaneous kriging-based sampling for optimization and uncertainty propagation. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov 2010

  • Jones DR, Pertunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theory Appl 79(1):157–181

    MathSciNet  MATH  Article  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492

    MathSciNet  MATH  Article  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surface. J Glob Optim 21:345–383

    MATH  Article  Google Scholar 

  • Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann Math Stat 23:462–466

    MathSciNet  MATH  Article  Google Scholar 

  • Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192:707–716

    MathSciNet  MATH  Article  Google Scholar 

  • Kleijnen JPC, van Beers W, Van Nieuwenhuyse I (2011) Expected improvement in efficient global optimization through bootstrapped kriging. Springer, pp 1–5

  • Lawler EL, Wood DE (1966) Branch and Bound methods: a survey. Oper Res 14(4):699–719

    MathSciNet  MATH  Article  Google Scholar 

  • Lucasius CB, Kateman G (1993) Understanding and using genetic algorithms Part 1. Concepts, properties and context. Chemometr Intell Lab Syst 19(1):1–33

    Article  Google Scholar 

  • Mathias K, Whitley D, Kusuma A, Stork C (1996) An empirical evaluation of genetic algorithms on noisy objective functions. CRC, pp 65–86

  • Nelder J, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313

    MATH  Article  Google Scholar 

  • Norkin V, Pflug GCh, Ruszczyński A (1996) A branch and bound method for stochastic global optimization. Math Program 83(1–3):452–450

    Google Scholar 

  • Picheny V, Ginsourger D, Richet Y (2010) Optimization of noisy computer experiments with tunable precision. Workshop on Noisy Kriging-based Optimization, (NKO Workshop), Bern, 22–24 Nov. 2010. Slides available at http://www.imsv.unibe.ch/content/continuingeducation/nko_workshop/program/index_ger.html

  • Robbins H, Monro S (1951) A Stochastic approximation method. Ann Math Stat 22:400–407

    MathSciNet  MATH  Article  Google Scholar 

  • Rullière D, Ribereau P (2011) Information aggregation and kriging alternative in a noisy environment. Preprint. French version available on HAL

  • Sakata S, Ashida F (2009) Ns-kriging based microstructural optimization applied to minimizing stochastic variation of homogenized elasticity of fiber reinforced composites. Struct Multidisc Optim 38:443–453

    Article  Google Scholar 

  • Schonlau M (1997) Computer experiments and global optimization. PhD, Dissertation, University of Waterloo

  • Schubert B (1972) A sequential method seeking the global maximum of a function. SIAM J Numer Anal 9:379–388

    MathSciNet  Article  Google Scholar 

  • Shin YS, Grandhi RV (2001) A global structural optimization technique using an interval method. Struct Multidisc Optim 22:351–363

    Article  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27:302–313

    Article  Google Scholar 

  • Smith NA, Tromble RW (2004) Sampling uniformly from the unit simplex. Technical Report, Johns Hopkins University

  • Strugarek C (2006) Approches variationnelles et autres contributions en optimisation stochastique. ENPC, Thèse de doctorat

    Google Scholar 

  • Vazquez E, Villemonteix J, Sidorkiewicz M, Walter E (2008) Global optimization based on noisy evaluations: an empirical study of two statistical approaches. J Phys Conference Series 135: 012100

    Google Scholar 

  • Villemonteix J (2009) Optimisation de fonctions coûteuses. Thèse de doctorat de physique, Université Paris Sud 11, Faculté des sciences d’Orsay, n9278

  • Woon SY, Querin OM, Steven GP (2001) Structural application of a shape optimization method based on a genetic algorithm. Struct Multidisc Optim 22:57–64

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and professor Ragnar Norberg for their valuable comments and suggestions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Didier Rullière.

Additional information

This work was presented in part at the Noisy Kriging-based Optimization (NKO) workshop, Bern, November 2010. It has been partially funded by ANR Research project ANR-08-BLAN-0314-01, by a grant from ANRT with reference 177/2008, by MIRACCLEGICC project, and Chaire BNP Paribas Cardif Management de la modélisation.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Rullière, D., Faleh, A., Planchet, F. et al. Exploring or reducing noise?. Struct Multidisc Optim 47, 921–936 (2013). https://doi.org/10.1007/s00158-012-0874-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-012-0874-5

Keywords

  • Global optimization
  • Noise
  • Potential
  • Branch-and-Bound
  • Simplex
  • Kriging