Skip to main content
SpringerLink
Log in

Efficient global optimization of constrained mixed variable problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Due to the increasing demand for high performance and cost reduction within the framework of complex system design, numerical optimization of computationally costly problems is an increasingly popular topic in most engineering fields. In this paper, several variants of the Efficient Global Optimization algorithm for costly constrained problems depending simultaneously on continuous decision variables as well as on quantitative and/or qualitative discrete design parameters are proposed. The adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product between the standard continuous kernel and a second kernel representing the covariance between the discrete variable values. Several parameterizations of this discrete kernel, with their respective strengths and weaknesses, are discussed in this paper. The novel algorithms are tested on a number of analytical test-cases and an aerospace related design problem, and it is shown that they require fewer function evaluations in order to converge towards the neighborhoods of the problem optima when compared to more commonly used optimization algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Agresti, A.: An Introduction to Categorical Data Analysis. Wiley, Hoboken (1996). https://ideas.repec.org/a/eee/csdana/v23y1997i4p565-563b.html

  2. Bajer, L., Holeňa, M.: Surrogate model for mixed-variables evolutionary optimization based on GLM and RBF networks. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7741 LNCS, pp. 481–490. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-35843-2_41

  3. Beauthier, C., Mahajan, A., Sainvitu, C., Hendrick, P., Sharifzadeh, S., Verstraete, D.: Hypersonic cryogenic tank design using mixed-variable surrogate-based optimization. In: Engineering Optimization IV—Proceedings of the 4th International Conference on Engineering Optimization, ENGOPT 2014, pp. 543–549. CRC Press (2014). https://doi.org/10.1201/b17488-98

  4. Dietrich, C.R., Osborne, M.R.: Estimation of covariance parameters in kriging via restricted maximum likelihood. Math. Geol. 23(1), 119–135 (1991). https://doi.org/10.1007/BF02065971

    Article  MathSciNet  MATH  Google Scholar 

  5. Durantin, C., Marzat, J., Balesdent, M.: Analysis of multi-objective kriging-based methods for constrained global optimization. Comput. Optim. Appl. 63(3), 903–926 (2016). https://doi.org/10.1007/s10589-015-9789-6

    Article  MathSciNet  MATH  Google Scholar 

  6. Fortin, F.A., De Rainville, F.M., Gardner, M.A., Parizeau, M., Gagné, C.: {DEAP}: Evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012)

    MathSciNet  MATH  Google Scholar 

  7. Gower, J.C.: A general coefficient of similarity and some of its properties. Biometrics 27(4), 857 (1971). https://doi.org/10.2307/2528823

    Article  Google Scholar 

  8. Haftka, R.T., Scott, E.P., Cruz, J.R.: Optimization and experiments: a survey. Appl. Mech. Rev. 51(7), 435–448 (1998). https://doi.org/10.1115/1.3099014

    Article  Google Scholar 

  9. Halstrup, M.: Black-Box Optimization of Mixed Discrete-Continuous Optimization Problems. Ph.D. Thesis, TU Dortmund (2016). https://doi.org/10.17877/DE290R-17800. https://eldorado.tu-dortmund.de/handle/2003/35773

  10. Hansen, N.: Towards a New Evolutionary Computation: Advances in the Estimation of Distribution Algorithms. Springer, Berlin (2006). https://doi.org/10.1007/3-540-32494-1_4

    Google Scholar 

  11. Holmström, K., Quttineh, N.H., Edvall, M.M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9, 311–339 (2008). https://doi.org/10.1007/s11081-008-9037-3

    Article  MathSciNet  MATH  Google Scholar 

  12. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  13. McKay, M.D., Beckman, R., Conover, W.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239 (1979). https://doi.org/10.2307/1268522

    MathSciNet  MATH  Google Scholar 

  14. Müller, J., Shoemaker, C.A., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Compute. Oper. Res. 40(5), 1383–1400 (2013). https://doi.org/10.1016/j.cor.2012.08.022

    Article  MathSciNet  MATH  Google Scholar 

  15. Oliver, M.A., Webster, R.: Kriging: a method of interpolation for geographical information systems. Int. J. Geogr. Inf. Syst. 4(3), 313–332 (1990). https://doi.org/10.1080/02693799008941549

    Article  Google Scholar 

  16. Pelamatti, J., Brevault, L., Balesdent, M., Talbi, E.G., Guerin, Y.: Overview and comparison of Gaussian process-based surrogate models for mixed continuous and discrete variables, application on aerospace design problems. In: High-Performance Simulation Based Optimization, Springer Series on Computational Intelligence (2018) (in Review)

  17. Pinheiro, J., Bates, D.: Mixed-effects models in S and S-PLUS. Stat. Comput. (2009). https://doi.org/10.1007/0-387-22747-4_8

  18. Pinheiro, J., Bates, D.M.: Unconstrained parametrizations for variance-covariance matrices. Stat. Comput. 6(3), 289–296 (1996). https://doi.org/10.1007/BF00140873

    Article  Google Scholar 

  19. Qian, P.Z.G., Wu, H., Wu, C.F.J.: Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics 50(3), 383–396 (2008). https://doi.org/10.1198/004017008000000262

    Article  MathSciNet  Google Scholar 

  20. Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidyanathan, R., Kevintucker, P.: Surrogate-based analysis and optimization. Prog. Aerosp. Sci. 41(1), 1–28 (2005). https://doi.org/10.1016/j.paerosci.2005.02.001

    Article  Google Scholar 

  21. Rashid, K., Ambani, S., Cetinkaya, E.: An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization. Eng. Optim. 45(2), 185–206 (2008). https://doi.org/10.1080/0305215X.2012.665450

    Article  MathSciNet  Google Scholar 

  22. Rasmussen, C.E., Williams, C.K.I.: Gaussian processes for machine learning. MIT Press (2006). http://www.gaussianprocess.org/gpml/

  23. Rebonato, R., Jaeckel, P.: The most general methodology to create a valid correlation matrix for risk management and option pricing purposes. SSRN Electron. J. (2011). https://doi.org/10.2139/ssrn.1969689

  24. Regis, R.G.: Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans. Evol. Comput. 18(3), 326–347 (2014). https://doi.org/10.1109/TEVC.2013.2262111

    Article  Google Scholar 

  25. Roustant, O., Padonou, E., Deville, Y., Clément, A., Perrin, G., Giorla, J., Wynn, H.: Group kernels for Gaussian process metamodels with categorical inputs (2018). https://hal-cea.archives-ouvertes.fr/hal-01702607v1. Accessed July 2018

  26. Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–423 (1989). https://doi.org/10.1214/ss/1177012413

    Article  MathSciNet  MATH  Google Scholar 

  27. Santner, T.J., Williams, B.J., Notz, W.I.: The Design and Analysis of Computer Experiments, p. 283. Springer, New York (2003)

  28. Sasena, M.J.: Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations. Ph.D. Thesis (2002)

  29. Schonlau, M., Welch, W.J., Jones, D.R.: Global versus local search in constrained optimization of computer models. Lect. Notes Monogr Ser 34, 11–25 (1998)

    Article  MathSciNet  Google Scholar 

  30. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, UK (2004)

    Book  MATH  Google Scholar 

  31. Simpson, T.W., Peplinski, J.D., Koch, P.N., Allen, J.K.: Metamodels for computer-based engineering design: survey and recommendations. Eng. Comput. 17(2), 129–150 (2001). https://doi.org/10.1007/PL00007198

    Article  MATH  Google Scholar 

  32. Stelmack, M., Nakashima, N., Batill, S.: Genetic algorithms for mixed discrete/continuous optimization in multidisciplinary design. In: 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization. American Institute of Aeronautics and Astronautics, Reston, Virigina (1998). https://doi.org/10.2514/6.1998-4771

  33. Swiler, L.P., Hough, P.D., Qian, P., Xu, X., Storlie, C., Lee, H.: Surrogate Models for Mixed Discrete-Continuous Variables. Springer International Publishing, Cham (2014). https://doi.org/10.1007/978-3-319-04280-0_21

  34. Wang, G., Shan, S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(4), 370 (2007). https://doi.org/10.1115/1.2429697

    Article  Google Scholar 

  35. Zhang, Y., Notz, W.I.: Computer experiments with qualitative and quantitative variables: a review and reexamination. Qual. Eng. 27(1), 2–13 (2015). https://doi.org/10.1080/08982112.2015.968039

    Article  Google Scholar 

  36. Zhou, Q., Qian, P.Z.G., Zhou, S.: A simple approach to emulation for computer models with qualitative and quantitative factors. Technometrics 53(3), 266–273 (2011). https://doi.org/10.1198/TECH.2011.10025

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre de Palaiseau, ONERA, 91120, Palaiseau, France

    Julien Pelamatti, Loïc Brevault & Mathieu Balesdent

  2. Inria Lille - Nord Europe, 59650, Villeneuve-d’Ascq, France

    El-Ghazali Talbi

  3. Centre national d’études spatiales Direction des lanceurs, 75012, Paris, France

    Julien Pelamatti & Yannick Guerin

Authors
  1. Julien Pelamatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Loïc Brevault
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathieu Balesdent
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. El-Ghazali Talbi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yannick Guerin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Pelamatti.

Additional information

This research is co-founded by the Centre National d’Études Spatiales (CNES) and by the Office National d’Études et de Recherches Aerospatiales (ONERA—The French Aerospace Lab) within the context of a Ph.D. thesis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pelamatti, J., Brevault, L., Balesdent, M. et al. Efficient global optimization of constrained mixed variable problems. J Glob Optim 73, 583–613 (2019). https://doi.org/10.1007/s10898-018-0715-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-018-0715-1

Keywords

Search

Navigation