Journal of Global Optimization

, Volume 55, Issue 4, pp 707–727 | Cite as

Worst-case global optimization of black-box functions through Kriging and relaxation

  • Julien Marzat
  • Eric Walter
  • Hélène Piet-Lahanier


A new algorithm is proposed to deal with the worst-case optimization of black-box functions evaluated through costly computer simulations. The input variables of these computer experiments are assumed to be of two types. Control variables must be tuned while environmental variables have an undesirable effect, to which the design of the control variables should be robust. The algorithm to be proposed searches for a minimax solution, i.e., values of the control variables that minimize the maximum of the objective function with respect to the environmental variables. The problem is particularly difficult when the control and environmental variables live in continuous spaces. Combining a relaxation procedure with Kriging-based optimization makes it possible to deal with the continuity of the variables and the fact that no analytical expression of the objective function is available in most real-case problems. Numerical experiments are conducted to assess the accuracy and efficiency of the algorithm, both on analytical test functions with known results and on an engineering application.


Computer experiments Continuous minimax Efficient global optimization Expected improvement Fault diagnosis Kriging Robust optimization Worst-case analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Santner T.J., Williams B.J., Notz W.: The Design and Analysis of Computer Experiments. Springer, Berlin, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Jones R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21(4), 345–383 (2001)CrossRefGoogle Scholar
  3. 3.
    Queipo N.V., Haftka R.T., Shyy W., Goel T., Vaidyanathan R., Tucker P.K.: Surrogate-based analysis and optimization. Prog. Aerosp. Sci. 41(1), 1–28 (2005)CrossRefGoogle Scholar
  4. 4.
    Simpson T.W., Poplinski J.D., Koch P.N., Allen J.K.: Metamodels for computer-based engineering design: survey and recommendations. Eng. Comput. 17(2), 129–150 (2001)CrossRefGoogle Scholar
  5. 5.
    McKay M.D., Beckman R.J., Conover W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979)Google Scholar
  6. 6.
    Matheron G.: Principles of geostatistics. Econ. Geol. 58(8), 1246–1266 (1963)CrossRefGoogle Scholar
  7. 7.
    Rasmussen C.E., Williams C.K.I.: Gaussian Processes for Machine Learning. Springer, New York, NY (2006)Google Scholar
  8. 8.
    Jones D.R., Schonlau M.J., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)CrossRefGoogle Scholar
  9. 9.
    Forrester A.I.J., Sobester A., Keane A.J.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, Chichester (2008)CrossRefGoogle Scholar
  10. 10.
    Huang D., Allen T.T., Notz W.I., Zeng N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. J. Glob. Optim. 34(3), 441–466 (2006)CrossRefGoogle Scholar
  11. 11.
    Sasena, M.J.: Flexibility and efficiency enhancements for constrained global design optimization with Kriging approximations. Ph.D. thesis, University of Michigan, USA (2002)Google Scholar
  12. 12.
    Villemonteix J., Vazquez E., Walter E.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Glob. Optim. 44(4), 509–534 (2009)CrossRefGoogle Scholar
  13. 13.
    Vazquez E., Bect J.: Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Stat. Plan. Inference 140(11), 3088–3095 (2010)CrossRefGoogle Scholar
  14. 14.
    Huang D., Allen T.T.: Design and analysis of variable fidelity experimentation applied to engine valve heat treatment process design. J. R. Stat. Soc. Ser. C Appl. Stat. 54(2), 443–463 (2005)CrossRefGoogle Scholar
  15. 15.
    Villemonteix, J., Vazquez, E., Walter, E.: Bayesian optimization for parameter identification on a small simulation budget. In: Proceedings of the 15th IFAC Symposium on System Identification, SYSID 2009, Saint-Malo France (2009)Google Scholar
  16. 16.
    Marzat, J., Walter, E., Piet-Lahanier, H., Damongeot, F.: Automatic tuning via Kriging-based optimization of methods for fault detection and isolation. In: Proceedings of the IEEE Conference on Control and Fault-Tolerant Systems, SYSTOL 2010, Nice, France, pp. 505–510 (2010)Google Scholar
  17. 17.
    Defretin, J., Marzat, J., Piet-Lahanier, H.: Learning viewpoint planning in active recognition on a small sampling budget: a Kriging approach. In: Proceedings of the 9th IEEE International Conference on Machine Learning and Applications, ICMLA 2010, Washington, USA, pp. 169–174 (2010)Google Scholar
  18. 18.
    Beyer H.G., Sendhoff B.: Robust optimization—a comprehensive survey. Comput. Methods Appl. Mech. Eng. 196(33–34), 3190–3218 (2007)CrossRefGoogle Scholar
  19. 19.
    Dellino G., Kleijnen J.P.C., Meloni C.: Robust optimization in simulation: Taguchi and response surface methodology. Int. J. Prod. Econ. 125(1), 52–59 (2010)CrossRefGoogle Scholar
  20. 20.
    Chen W., Allen J.K., Tsui K.L., Mistree F.: A procedure for robust design: minimizing variations caused by noise factors and control factors. ASME J. Mech. Des. 118, 478–485 (1996)CrossRefGoogle Scholar
  21. 21.
    Lee, K., Park, G., Joo, W.: A global robust optimization using the Kriging based approximation model. In: Proceedings of the 6th World Congresses of Structural and Multidisciplinary Optimization, Rio de Janeiro, Brazil (2005)Google Scholar
  22. 22.
    Williams B.J., Santner T.J., Notz W.I.: Sequential design of computer experiments to minimize integrated response functions. Statistica Sinica 10(4), 1133–1152 (2000)Google Scholar
  23. 23.
    Lehman J.S., Santner T.J., Notz W.I.: Designing computer experiments to determine robust control variables. Statistica Sinica 14(2), 571–590 (2004)Google Scholar
  24. 24.
    Lam, C.Q.: Sequential adaptive designs in computer experiments for response surface model fit. Ph.D. thesis, The Ohio State University (2008)Google Scholar
  25. 25.
    Cramer A.M., Sudhoff S.D., Zivi E.L.: Evolutionary algorithms for minimax problems in robust design. IEEE Trans. Evolut. Comput. 13(2), 444–453 (2009)CrossRefGoogle Scholar
  26. 26.
    Lung, R.I., Dumitrescu, D.: A new evolutionary approach to minimax problems. In: Proceedings of the 2011 IEEE Congress on Evolutionary Computation, New Orleans, USA, pp. 1902–1905 (2011)Google Scholar
  27. 27.
    Zhou, A., Zhang, Q.: A surrogate-assisted evolutionary algorithm for minimax optimization. In: Proceedings of the 2010 IEEE Congress on Evolutionary Computation, Barcelona, Spain, pp. 1–7 (2010)Google Scholar
  28. 28.
    Shimizu K., Aiyoshi E.: Necessary conditions for min-max problems and algorithms by a relaxation procedure. IEEE Trans. Autom. Control 25(1), 62–66 (1980)CrossRefGoogle Scholar
  29. 29.
    Rustem B., Howe M.: Algorithms for Worst-Case Design and Applications to Risk Management. Princeton University Press, Princeton, NJ (2002)Google Scholar
  30. 30.
    Brown B., Singh T.: Minimax design of vibration absorbers for linear damped systems. J. Sound Vib. 330(11), 2437–2448 (2011)CrossRefGoogle Scholar
  31. 31.
    Salmon D.M.: Minimax controller design. IEEE Trans. Autom. Control 13(4), 369–376 (1968)CrossRefGoogle Scholar
  32. 32.
    Helton J.: Worst case analysis in the frequency domain: the H approach to control. IEEE Trans. Autom. Control 30(12), 1154–1170 (1985)CrossRefGoogle Scholar
  33. 33.
    Chow E.Y., Willsky A.S.: Analytical redundancy and the design of robust failure detection systems. IEEE Trans. Autom. Control 29, 603–614 (1984)CrossRefGoogle Scholar
  34. 34.
    Frank P.M., Ding X.: Survey of robust residual generation and evaluation methods in observer-based fault detection systems. J. Process Control 7(6), 403–424 (1997)CrossRefGoogle Scholar
  35. 35.
    Colson B., Marcotte P., Savard G.: An overview of bilevel optimization. Ann. Oper. Res. 153(1), 235–256 (2007)CrossRefGoogle Scholar
  36. 36.
    Ben-Tal A., Nemirovski A.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998)CrossRefGoogle Scholar
  37. 37.
    Başar T., Olsder G.J.: Dynamic Noncooperative Game Theory. Society for Industrial Mathematics, New York, NY (1999)Google Scholar
  38. 38.
    Du D., Pardalos P.M.: Minimax and Applications. Kluwer, Norwell (1995)CrossRefGoogle Scholar
  39. 39.
    Parpas P., Rustem B.: An algorithm for the global optimization of a class of continuous minimax problems. J. Optim. Theory Appl. 141(2), 461–473 (2009)CrossRefGoogle Scholar
  40. 40.
    Tsoukalas A., Rustem B., Pistikopoulos E.N.: A global optimization algorithm for generalized semi-infinite, continuous minimax with coupled constraints and bi-level problems. J. Glob. Optim. 44(2), 235–250 (2009)CrossRefGoogle Scholar
  41. 41.
    Rustem B.: Algorithms for Nonlinear Programming and Multiple Objective Decisions. Wiley, Chichester (1998)Google Scholar
  42. 42.
    Shimizu K., Ishizuka Y., Bard J.F.: Nondifferentiable and Two-level Mathematical Programming. Kluwer, Norwell (1997)CrossRefGoogle Scholar
  43. 43.
    MacKay D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge, MA (2003)Google Scholar
  44. 44.
    Schonlau, M.: Computer experiments and global optimization. Ph.D. thesis, University of Waterloo, Canada (1997)Google Scholar
  45. 45.
    Jones D.R., Perttunen C.D., Stuckman B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)CrossRefGoogle Scholar
  46. 46.
    Basseville M., Nikiforov I.V.: Detection of Abrupt Changes: Theory and Application. Prentice Hall, Englewood Cliffs, NJ (1993)Google Scholar
  47. 47.
    Bartyś M., Patton R.J., Syfert M., delas Heras S., Quevedo J.: Introduction to the DAMADICS actuator FDI benchmark study. Control Eng. Pract. 14(6), 577–596 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2012

Authors and Affiliations

  • Julien Marzat
    • 1
    • 2
  • Eric Walter
    • 2
  • Hélène Piet-Lahanier
    • 1
  1. 1.ONERA (The French Aerospace Lab)PalaiseauFrance
  2. 2.Gif-sur-YvetteFrance

Personalised recommendations