Skip to main content
SpringerLink
Log in

Worst-case global optimization of black-box functions through Kriging and relaxation

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

A new algorithm is proposed to deal with the worst-case optimization of black-box functions evaluated through costly computer simulations. The input variables of these computer experiments are assumed to be of two types. Control variables must be tuned while environmental variables have an undesirable effect, to which the design of the control variables should be robust. The algorithm to be proposed searches for a minimax solution, i.e., values of the control variables that minimize the maximum of the objective function with respect to the environmental variables. The problem is particularly difficult when the control and environmental variables live in continuous spaces. Combining a relaxation procedure with Kriging-based optimization makes it possible to deal with the continuity of the variables and the fact that no analytical expression of the objective function is available in most real-case problems. Numerical experiments are conducted to assess the accuracy and efficiency of the algorithm, both on analytical test functions with known results and on an engineering application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Santner T.J., Williams B.J., Notz W.: The Design and Analysis of Computer Experiments. Springer, Berlin, Heidelberg (2003)

    Book  Google Scholar 

  2. Jones R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21(4), 345–383 (2001)

    Article  Google Scholar 

  3. Queipo N.V., Haftka R.T., Shyy W., Goel T., Vaidyanathan R., Tucker P.K.: Surrogate-based analysis and optimization. Prog. Aerosp. Sci. 41(1), 1–28 (2005)

    Article  Google Scholar 

  4. Simpson T.W., Poplinski J.D., Koch P.N., Allen J.K.: Metamodels for computer-based engineering design: survey and recommendations. Eng. Comput. 17(2), 129–150 (2001)

    Article  Google Scholar 

  5. McKay M.D., Beckman R.J., Conover W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979)

    Google Scholar 

  6. Matheron G.: Principles of geostatistics. Econ. Geol. 58(8), 1246–1266 (1963)

    Article  Google Scholar 

  7. Rasmussen C.E., Williams C.K.I.: Gaussian Processes for Machine Learning. Springer, New York, NY (2006)

    Google Scholar 

  8. Jones D.R., Schonlau M.J., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)

    Article  Google Scholar 

  9. Forrester A.I.J., Sobester A., Keane A.J.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, Chichester (2008)

    Book  Google Scholar 

  10. Huang D., Allen T.T., Notz W.I., Zeng N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. J. Glob. Optim. 34(3), 441–466 (2006)

    Article  Google Scholar 

  11. Sasena, M.J.: Flexibility and efficiency enhancements for constrained global design optimization with Kriging approximations. Ph.D. thesis, University of Michigan, USA (2002)

  12. Villemonteix J., Vazquez E., Walter E.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Glob. Optim. 44(4), 509–534 (2009)

    Article  Google Scholar 

  13. Vazquez E., Bect J.: Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Stat. Plan. Inference 140(11), 3088–3095 (2010)

    Article  Google Scholar 

  14. Huang D., Allen T.T.: Design and analysis of variable fidelity experimentation applied to engine valve heat treatment process design. J. R. Stat. Soc. Ser. C Appl. Stat. 54(2), 443–463 (2005)

    Article  Google Scholar 

  15. Villemonteix, J., Vazquez, E., Walter, E.: Bayesian optimization for parameter identification on a small simulation budget. In: Proceedings of the 15th IFAC Symposium on System Identification, SYSID 2009, Saint-Malo France (2009)

  16. Marzat, J., Walter, E., Piet-Lahanier, H., Damongeot, F.: Automatic tuning via Kriging-based optimization of methods for fault detection and isolation. In: Proceedings of the IEEE Conference on Control and Fault-Tolerant Systems, SYSTOL 2010, Nice, France, pp. 505–510 (2010)

  17. Defretin, J., Marzat, J., Piet-Lahanier, H.: Learning viewpoint planning in active recognition on a small sampling budget: a Kriging approach. In: Proceedings of the 9th IEEE International Conference on Machine Learning and Applications, ICMLA 2010, Washington, USA, pp. 169–174 (2010)

  18. Beyer H.G., Sendhoff B.: Robust optimization—a comprehensive survey. Comput. Methods Appl. Mech. Eng. 196(33–34), 3190–3218 (2007)

    Article  Google Scholar 

  19. Dellino G., Kleijnen J.P.C., Meloni C.: Robust optimization in simulation: Taguchi and response surface methodology. Int. J. Prod. Econ. 125(1), 52–59 (2010)

    Article  Google Scholar 

  20. Chen W., Allen J.K., Tsui K.L., Mistree F.: A procedure for robust design: minimizing variations caused by noise factors and control factors. ASME J. Mech. Des. 118, 478–485 (1996)

    Article  Google Scholar 

  21. Lee, K., Park, G., Joo, W.: A global robust optimization using the Kriging based approximation model. In: Proceedings of the 6th World Congresses of Structural and Multidisciplinary Optimization, Rio de Janeiro, Brazil (2005)

  22. Williams B.J., Santner T.J., Notz W.I.: Sequential design of computer experiments to minimize integrated response functions. Statistica Sinica 10(4), 1133–1152 (2000)

    Google Scholar 

  23. Lehman J.S., Santner T.J., Notz W.I.: Designing computer experiments to determine robust control variables. Statistica Sinica 14(2), 571–590 (2004)

    Google Scholar 

  24. Lam, C.Q.: Sequential adaptive designs in computer experiments for response surface model fit. Ph.D. thesis, The Ohio State University (2008)

  25. Cramer A.M., Sudhoff S.D., Zivi E.L.: Evolutionary algorithms for minimax problems in robust design. IEEE Trans. Evolut. Comput. 13(2), 444–453 (2009)

    Article  Google Scholar 

  26. Lung, R.I., Dumitrescu, D.: A new evolutionary approach to minimax problems. In: Proceedings of the 2011 IEEE Congress on Evolutionary Computation, New Orleans, USA, pp. 1902–1905 (2011)

  27. Zhou, A., Zhang, Q.: A surrogate-assisted evolutionary algorithm for minimax optimization. In: Proceedings of the 2010 IEEE Congress on Evolutionary Computation, Barcelona, Spain, pp. 1–7 (2010)

  28. Shimizu K., Aiyoshi E.: Necessary conditions for min-max problems and algorithms by a relaxation procedure. IEEE Trans. Autom. Control 25(1), 62–66 (1980)

    Article  Google Scholar 

  29. Rustem B., Howe M.: Algorithms for Worst-Case Design and Applications to Risk Management. Princeton University Press, Princeton, NJ (2002)

    Google Scholar 

  30. Brown B., Singh T.: Minimax design of vibration absorbers for linear damped systems. J. Sound Vib. 330(11), 2437–2448 (2011)

    Article  Google Scholar 

  31. Salmon D.M.: Minimax controller design. IEEE Trans. Autom. Control 13(4), 369–376 (1968)

    Article  Google Scholar 

  32. Helton J.: Worst case analysis in the frequency domain: the H ∞ approach to control. IEEE Trans. Autom. Control 30(12), 1154–1170 (1985)

    Article  Google Scholar 

  33. Chow E.Y., Willsky A.S.: Analytical redundancy and the design of robust failure detection systems. IEEE Trans. Autom. Control 29, 603–614 (1984)

    Article  Google Scholar 

  34. Frank P.M., Ding X.: Survey of robust residual generation and evaluation methods in observer-based fault detection systems. J. Process Control 7(6), 403–424 (1997)

    Article  Google Scholar 

  35. Colson B., Marcotte P., Savard G.: An overview of bilevel optimization. Ann. Oper. Res. 153(1), 235–256 (2007)

    Article  Google Scholar 

  36. Ben-Tal A., Nemirovski A.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998)

    Article  Google Scholar 

  37. Başar T., Olsder G.J.: Dynamic Noncooperative Game Theory. Society for Industrial Mathematics, New York, NY (1999)

    Google Scholar 

  38. Du D., Pardalos P.M.: Minimax and Applications. Kluwer, Norwell (1995)

    Book  Google Scholar 

  39. Parpas P., Rustem B.: An algorithm for the global optimization of a class of continuous minimax problems. J. Optim. Theory Appl. 141(2), 461–473 (2009)

    Article  Google Scholar 

  40. Tsoukalas A., Rustem B., Pistikopoulos E.N.: A global optimization algorithm for generalized semi-infinite, continuous minimax with coupled constraints and bi-level problems. J. Glob. Optim. 44(2), 235–250 (2009)

    Article  Google Scholar 

  41. Rustem B.: Algorithms for Nonlinear Programming and Multiple Objective Decisions. Wiley, Chichester (1998)

    Google Scholar 

  42. Shimizu K., Ishizuka Y., Bard J.F.: Nondifferentiable and Two-level Mathematical Programming. Kluwer, Norwell (1997)

    Book  Google Scholar 

  43. MacKay D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge, MA (2003)

    Google Scholar 

  44. Schonlau, M.: Computer experiments and global optimization. Ph.D. thesis, University of Waterloo, Canada (1997)

  45. Jones D.R., Perttunen C.D., Stuckman B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)

    Article  Google Scholar 

  46. Basseville M., Nikiforov I.V.: Detection of Abrupt Changes: Theory and Application. Prentice Hall, Englewood Cliffs, NJ (1993)

    Google Scholar 

  47. Bartyś M., Patton R.J., Syfert M., delas Heras S., Quevedo J.: Introduction to the DAMADICS actuator FDI benchmark study. Control Eng. Pract. 14(6), 577–596 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ONERA (The French Aerospace Lab), 91761, Palaiseau, France

    Julien Marzat & Hélène Piet-Lahanier

  2. L2S, CNRS-SUPELEC-Univ Paris-Sud, 91192, Gif-sur-Yvette, France

    Julien Marzat & Eric Walter

Authors
  1. Julien Marzat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Walter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hélène Piet-Lahanier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Marzat.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marzat, J., Walter, E. & Piet-Lahanier, H. Worst-case global optimization of black-box functions through Kriging and relaxation. J Glob Optim 55, 707–727 (2013). https://doi.org/10.1007/s10898-012-9899-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-012-9899-y

Keywords

Search

Navigation