Skip to main content

Bayesian optimization of variable-size design space problems

Abstract

Within the framework of complex system design, it is often necessary to solve mixed variable optimization problems, in which the objective and constraint functions can depend simultaneously on continuous and discrete variables. Additionally, complex system design problems occasionally present a variable-size design space. This results in an optimization problem for which the search space varies dynamically (with respect to both number and type of variables) along the optimization process as a function of the values of specific discrete decision variables. Similarly, the number and type of constraints can vary as well. In this paper, two alternative Bayesian optimization-based approaches are proposed in order to solve this type of optimization problems. The first one consists of a budget allocation strategy allowing to focus the computational budget on the most promising design sub-spaces. The second approach, instead, is based on the definition of a kernel function allowing to compute the covariance between samples characterized by partially different sets of variables. The results obtained on analytical and engineering related test-cases show a faster and more consistent convergence of both proposed methods with respect to the standard approaches.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org

  • Abdelkhalik O (2013) Autonomous planning of multigravity-assist trajectories with deep space maneuvers using a differential evolution approach. Int J Aerosp Eng, 1–11

  • Abramson MA, Audet C, Chrissis JW, Walston JG (2009) Mesh adaptive direct search algorithms for mixed variable optimization. Optim Lett 3(1):35–47

    MathSciNet  Article  Google Scholar 

  • Abramson MA, Audet C, Dennis JE Jr (2007) Filter pattern search algorithms for mixed variable constrained optimization problems. Pac J Optim 3(3):477–500

    MathSciNet  MATH  Google Scholar 

  • Alvarez MA, Rosasco L, Lawrence ND et al (2012) Kernels for vector-valued functions: a review. Found Trends Mach Learn 4(3):195–266

    Article  Google Scholar 

  • Audet C, Dennis JE Jr (2000) Pattern search algorithms for mixed variable programming. SIAM J Optim 11(3):573–594

    MathSciNet  Article  Google Scholar 

  • Audet C, Denni J, Moore D, Booker A, Frank P (2000) A surrogate-model-based method for constrained optimization. In: 8th symposium on multidisciplinary analysis and optimization

  • Byrd Richard H, Peihuang L, Jorge N, Ciyou Z (1995) A limited memory algorithm for bound constrained optimization. SIAM J Sci Comput 16(5):1190–1208

    MathSciNet  Article  Google Scholar 

  • Cédric D, Julien M, Mathieu B (2016) Analysis of multi-objective Kriging-based methods for constrained global optimization. Comput Optim Appl 63(3):903–926

    MathSciNet  Article  Google Scholar 

  • de Matthews AG, van der Wilk M, Nickson T, Fujii K, Boukouvalas A, León-Villagrá P, Ghahramani Z, Hensman J (2017) GPflow: a Gaussian process library using TensorFlow. J Mach Learn Res 18(40):1–6

    MathSciNet  MATH  Google Scholar 

  • Filomeno Coelho Rajan (2014) Metamodels for mixed variables based on moving least squares. Optim Eng 15(2):311–329

    MathSciNet  Article  Google Scholar 

  • Frank CP (2016) A design space exploration methodology to support decisions under evolving uncertainty in requirements and its application to advanced vehicles

  • Frazier PI, Clark SC (2012) Parallel global optimization using an improved multi-points expected improvement criterion. In: INFORMS optimization society conference, Miami FL, vol 26

  • Félix-Antoine F, Marc-André DRF-MG, Marc P, Christian G (2012) DEAP: evolutionary algorithms made easy. J Mach Learn Res 13:2171–2175

    MathSciNet  Google Scholar 

  • Ginsbourger D, Le Riche R, Carraro L (2010) Kriging is well-suited to parallelize optimization. In: Computational intelligence in expensive optimization problems, Springer, BErlin, pp 131–162

  • Girdziušas R, Janusevskis J, Le Riche R (2012) On integration of multi-point improvements

  • Goldberg DE (1989) Genetic algorithms in search, optimization & machine learning. Addison-Wesley Longman Publishing Co., Inc, New York

    MATH  Google Scholar 

  • Halstrup M (2016) Black-box optimization of mixed discrete-continuous optimization problems. PhD thesis, TU Dortmund

  • Herbrich R, Lawrence ND, Seeger M (2003) Fast sparse gaussian process methods: the informative vector machine. In: Advances in neural information processing systems, pp 625–632

  • HoHo R (1960) An automatic method for finding the greatest or least value of a function. Comput J 3(3):175–184

    MathSciNet  Article  Google Scholar 

  • Horn D, Stork J, Schüßler N-J, Zaefferer M (2019) Surrogates for hierarchical search spaces: the wedge-kernel and an automated analysis. In: Proceedings of the genetic and evolutionary computation conference, pp 916–924

  • Hutter F (2009) Automated configuration of algorithms for solving hard computational problems. PhD thesis, University of British Columbia

  • Hutter F, Osborne MA (2013) A kernel for hierarchical parameter spaces. arXiv preprint arXiv:1310.5738

  • Jonas M (1994) Application of bayesian approach to numerical methods of global and stochastic optimization. J Global Optim 4(4):347–365

    MathSciNet  Article  Google Scholar 

  • Jones DR, Matthias S, Welch William J (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13:455–492

    MathSciNet  Article  Google Scholar 

  • Julien P, Loïc B, Mathieu B, El-Ghazali T, Yannick G (2018) How to deal with mixed-variable optimization problems: an overview of algorithms and formulations. Advances in structural and multidisciplinary optimization. Springer International Publishing, Cham, pp 64–82

    Google Scholar 

  • Julien P, Loïc B, Mathieu B, El-Ghazali T, Yannick G (2019) Efficient global optimization of constrained mixed variable problems. J Global Optim 73(3):583–613

    MathSciNet  Article  Google Scholar 

  • Land AH, Doig AG (2010) An automatic method for solving discrete programming problems. 50 years of integer programming 1958–2008. Springer, Berlin, pp 105–132

    Chapter  Google Scholar 

  • Lehel C, Manfred O (2002) Sparse on-line gaussian processes. Neural Comput 14(3):641–668

    Article  Google Scholar 

  • McKay MD, Richard B, William C (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239

    MathSciNet  Google Scholar 

  • Meen NH, Ossama A, Nilufer O (2015) Structured-chromosome evolutionary algorithms for variable-size autonomous interplanetary trajectory planning optimization. J Aerosp Inf Syst 12(3):314–328

    Google Scholar 

  • Minasny B, McBratney AB (2005) The Matérn function as a general model for soil variograms. In: Geoderma, Elsevier, Amsterdam, vol 128, pp 192–207

  • Nachman A (1950) Theory of reproducing kernels. Trans Am Math Soc 68(3):337–404

    MathSciNet  Article  Google Scholar 

  • Nelder John A, Roger M (1964) A simplex method for function minimization. Comput J 7(4):308–313

    MathSciNet  Article  Google Scholar 

  • Ossama A (2013) Hidden genes genetic optimization for variable-size design space problems. J Optim Theory Appl 156(2):450–468

    MathSciNet  Article  Google Scholar 

  • Pelamatti J, Brevault L, Balesdent M, Talbi E, Guerin Y (2019) Surrogate model based optimization of constrained mixed variable problems: application to the design of a launch vehicle thrust frame. In: AIAA Scitech 2019 Forum

  • Pelamatti J, Brevault L, Balesdent M, Talbi E, Guerin Y (2020) Overview and comparison of gaussian process-based surrogate models for mixed continuous and discrete variables: application on aerospace design problems. In: High-performance simulation-based optimization, Springer, Berlin, pp 189–224

  • Prasadh N, Moss R, Collett K, Nelessen A, Edwards S, Mavris DN (2014) A systematic method for sme-driven space system architecture down-selection. In: AIAA SPACE 2014 conference and exposition, pp 4–7

  • Priem R, Bartoli N, Diouane Y (2019) On the use of upper trust bounds in constrained bayesian optimization infill criteria. In: AIAA aviation 2019 forum, p 2986

  • Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge

    MATH  Google Scholar 

  • Roustant O, Padonou E, Deville Y, Clément A, Perrin G, Giorla J, Wynn H (2018) Group kernels for gaussian process metamodels with categorical inputs. arXiv preprint arXiv:1802.02368

  • Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer, New York

    Book  Google Scholar 

  • Sasena MJ (2002) Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. PhD thesis

  • Scholkopf B, Smola AJ (2001) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, Cambridge

    Google Scholar 

  • Simpson Timothy W, Peplinski JD, Koch Patrick N, Allen Janet K (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150

    Article  Google Scholar 

  • Stefano L, Veronica P, Marco S (2005) An algorithm model for mixed variable programming. SIAM J Optim 15(4):1057–1084

    MathSciNet  Article  Google Scholar 

  • Steinwart I, Christmann A (2008) Support vector machines. Springer Science & Business Media, Berlin

    MATH  Google Scholar 

  • Stelmack M, Nakashima N, Batill S (1998) Genetic algorithms for mixed discrete/continuous optimization in multidisciplinary design. In: 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, Reston, Virigina, sep 1998. American Institute of Aeronautics and Astronautics

  • Swiler LP, Hough PD, Qian P, Xu X, Storlie C, Lee H (2014) Surrogate models for mixed discrete-continuous variables. In: Constraint programming and decision making, Springer, Berlin, pp 181–202

  • Victor P, Tobias W, David G (2013) A benchmark of kriging-based infill criteria for noisy optimization. Struct Multidiscipl Optim 48(3):607–626

    Article  Google Scholar 

  • Wertz JR (2001) Mission geometry: orbit and constellation design and management: spacecraft orbit and attitude systems. Mission geometry: orbit and constellation design and management: spacecraft orbit and attitude systems/James R. Wertz. El Segundo, CA; Boston: Microcosm: Kluwer Academic Publishers, 2001. Space technology library; 13

  • Yichi Z, Apley Daniel W, Wei C (2020) Bayesian optimization for materials design with mixed quantitative and qualitative variables. Sci Rep 10(1):1–13

    Article  Google Scholar 

  • Zaefferer M, Horn D (2018) A first analysis of kernels for kriging-based optimization in hierarchical search spaces. In: International conference on parallel problem solving from nature, pp 399–410. Springer, Berlin

  • Zhang Y, Notz WI (2015) Computer experiments with qualitative and quantitative variables: a review and reexamination. In: Quality engineering, Taylor & Francis, vol 27, pp 2–13

  • Zhang Y, Tao S, Chen W, Apley DW (2019) A latent variable approach to gaussian process modeling with qualitative and quantitative factors. Technometrics, pp 1–12

  • Zhou Q, Qian PZG, Zhou S (2011) A simple approach to emulation for computer models with qualitative and quantitative factors. Technometrics 53(3):266–273

    MathSciNet  Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Julien Pelamatti.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research is founded by the Centre National d’Études Spatiales (CNES) and by the Office National d’ Études et de Recherches Aerospatiales (ONERA–The French Aerospace Lab) within the context of a PhD thesis.

Appendices

Appendix A: Characteristics of the text-case and implementation

Variable-size design space Goldstein function

  • 5 continuous variables, 4 discrete variables, 2 dimensional variables

  • 8 sub-problems

  • 648 equivalent continuous problems

  • 1 constraint

  • Initial data set size: 104 samples ( e.g. 2 samples per dimension of each sub-problem)

  • Number of infilled samples: 104

  • 10 repetitions

  • Compared methods:

    • Independent mixed-variable BO of each sub-problem (IO) with both CS and LV kernels

    • SOMVSP (BA) with CS and LV kernels and values of a of 2 and 3

    • Variable-size design space kernel BO with CS and LV kernels with both SPW and DVW approaches

  • Acquisition function: EI under EV constraints

Variable-size design space Rosenbrock function

  • 8 continuous variables, 3 discrete variables, 2 dimensional variables

  • 4 sub-problems

  • 32 equivalent continuous problems

  • 2 constraints

  • Initial data set size: 30 samples (e.g. 1 sample per dimension of each sub-problem)

  • Number of infilled samples: 65

  • 10 repetitions

  • Compared methods:

    • Independent mixed-variable BO of each sub-problem (IO) with both CS and LV kernels

    • SOMVSP (BA) with CS and LV kernels and values of a of 2 and 3

    • Variable-size design space kernel BO with CS and LV kernels with both SPW and DVW approaches

  • Acquisition function: EI under EV constraints

Multi-stage launch vehicle design

  • 18 continuous variables, 14 discrete variables, 3 dimensional variables

  • 6 sub-problems

  • 29136 equivalent continuous problems

  • 19 constraints

  • Initial data set size: 122 samples (e.g. 1.5 sample per dimension of each sub-problem)

  • Number of infilled samples: 58

  • 10 repetitions

  • Compared methods:

    • Independent mixed-variable BO of each sub-problem (IO) with both CS and LV kernels

    • SOMVSP (BA) with CS and LV kernels and values of a of 3

    • Variable-size design space kernel BO with CS and LV kernels with both SPW and DVW approaches

  • Acquisition function: EI under EV constraints

  • Large number of continuous and discrete design variables. Large number of constraints.

Implementation

The results presented in the paper are obtained with the following implementation. The optimization routine overhead is written in Python 3.6. The GP models are created with the help of GPflow (Alexander et al. 2017), a Python-based toolbox for GP-based modeling relying on the Tensorflow framework (Abadi et al. 2015) (version 1.13). The surrogate model training is performed with the help of a Bounded Limited memory Broyden–Fletcher–Goldfarb Shanno (L-BFGS-B) algorithm (Byrd et al. 1995), whereas the acquisition functions are optimized the help of a constraint domination based mixed continuous/discrete genetic algorithm (Stelmack et al. 1998) implemented by relying on the Python-based toolbox DEAP (Fortin et al. 2012).

Appendix B: Variable-size design space Goldstein function

The variable-size design space variant of the Goldstein function which is considered for the testing discussed in Sect. 5 is characterized by a global design space with 5 continuous design variables, 4 discrete design variables and 2 dimensional design variables. Depending on the dimensional variable values, 8 different sub-problems can be identified, with total dimensions of 6 or 7, ranging from 2 continuous variables and 4 discrete variables to 5 continuous variables and 2 discrete variables. All of the sub-problem are subject to a variable-size design space constraint.

The resulting optimization problem can be defined as follows:

$$\begin{aligned} \min&\qquad f(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) \nonumber \\ \text {w.r.t.}&\qquad \mathbf{x} = \{x_1,\dots ,x_5\} \ \text{ with } \ x_i \in [0,100] \ \text{ for } i = 1,5 \nonumber \\&\qquad \mathbf{z} = \{z_1,\dots ,z_4\} \ \text{ with } \ z_i \in \{ 0,1,2\} \ \text{ for } i = 1,4 \nonumber \\&\qquad \mathbf{w} = \{w_1,w_2\} \ \text{ with } \ w_1 \in \{ 0,1,2,3\} \ \text{ and } w_2 \in \{ 0,1\} \nonumber \\ \text {s.t.:}&\qquad g(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) \le 0 \end{aligned}$$
(55)

where

$$\begin{aligned} f(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) = {\left\{ \begin{array}{ll} f_1(x_1,x_2,z_1,z_2,z_3,z_4) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 0\\ f_2(x_1,x_2,x_3,z_2,z_3,z_4) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 0\\ f_3(x_1,x_2,x_4,z_1,z_3,z_4) \qquad \text{ if } w_1 = 2 \text{ and } w_2 = 0\\ f_4(x_1,x_2,x_3,x_4,z_3,z_4) \qquad \text{ if } w_1 = 3 \text{ and } w_2 = 0\\ f_5(x_1,x_2,x_5,z_1,z_2,z_3,z_4) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 1\\ f_6(x_1,x_2,x_3,x_5,z_2,z_3,z_4) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 1\\ f_7(x_1,x_2,x_4,x_5,z_1,z_3,z_4) \qquad \text{ if } w_1 = 2 \text{ and } w_2 = 1\\ f_8(x_1,x_2,x_3,x_5,x_4,z_3,z_4) \qquad \text{ if } w_1 = 3 \text{ and } w_2 = 1 \end{array}\right. } \end{aligned}$$
(56)

and

$$\begin{aligned} g(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) = {\left\{ \begin{array}{ll} g_1(x_1,x_2,z_1,z_2) \qquad \text{ if } w_1 = 0 \\ g_2(x_1,x_2,z_2) \qquad \text{ if } w_1 = 1 \\ g_3(x_1,x_2,z_1) \qquad \text{ if } w_1 = 2 \\ g_4(x_1,x_2,z_3,z_4) \qquad \text{ if } w_1 = 3 \end{array}\right. } \end{aligned}$$
(57)

The objective functions \(f_1(\cdot ),\dots ,f_8(\cdot )\) are defined as follows:

$$\begin{aligned} \begin{aligned} f_1(x_1,x_2,z_1,z_2,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} \end{aligned} \end{aligned}$$
(58)

where \(x_3\) and \(x_4\) are defined as a function of \(z_1\) and \(z_2\) according to the relations defined in Table 6.

Table 6 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)1 discrete categories
$$\begin{aligned} \begin{aligned} f_2(x_1,x_2,x_3,z_2,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} \end{aligned} \end{aligned}$$
(59)

where \(x_4\) is defined as a function of \(z_2\) according to the relations defined in Table 7.

Table 7 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)2 discrete categories
$$\begin{aligned} \begin{aligned} f_3(x_1,x_2,x_4,z_1,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} \end{aligned} \end{aligned}$$
(60)

where \(x_3\) is defined as a function of \(z_1\) according to the relations defined in Table 8.

Table 8 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)2 discrete categories
$$\begin{aligned}&\begin{aligned} f_4(x_1,x_2,x_3,x_4,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} \end{aligned} \end{aligned}$$
(61)
$$\begin{aligned}&\begin{aligned} f_5(x_1,x_2,z_1,z_2,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} + 5 \cos (2 \pi \frac{x_5}{100})-2 \end{aligned} \end{aligned}$$
(62)

where \(x_3\) and \(x_4\) are defined as a function of \(z_1\) and \(z_2\) according to the relations defined in Table 9.

Table 9 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)5 discrete categories
$$\begin{aligned} \begin{aligned} f_6(x_1,x_2,x_3,z_2,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} + 5 \cos (2 \pi \frac{x_5}{100})-2 \end{aligned} \end{aligned}$$
(63)

where \(x_4\) is defined as a function of \(z_2\) according to the relations defined in Table 10.

Table 10 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)6 discrete categories
$$\begin{aligned} \begin{aligned} f_7(x_1,x_2,x_4,z_1,z_3,z_4) = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} + 5 \cos (2 \pi \frac{x_5}{100})-2 \end{aligned} \end{aligned}$$
(64)

where \(x_3\) is defined as a function of \(z_1\) according to the relations defined in Table 11.

Table 11 Characterization of the variable-dimension search space Goldstein function sub-problem N\({}^\circ \)7 discrete categories
$$\begin{aligned} \begin{aligned} f_8(x_1,x_2,x_3,x_4,z_3,z_4 = \quad&53.3108+ 0.184901x_1-5.02914 x_1^3 \cdot 10^{-6} + 7.72522x_1^{z_3} \cdot 10^{-8}- \\&0.0870775x_2 - 0.106959 x_3 + 7.98772x_3^{z_4} \cdot 10^{-6} + \\&0.00242482 x_4 + 1.32851 x_4^3 \cdot 10^{-6} - 0.00146393 x_1 x_2 - \\&0.00301588 x_1 x_3 - 0.00272291 x_1 x_4+ 0.0017004 x_2 x_3 + \\&0.0038428 x_2 x_4 - 0.000198969 x_3x_4 + 1.86025 x_1x_2x_3 \cdot 10^{-5} - \\&1.88719 x_1 x_2 x_4 \cdot 10^{-6}+ 2.50923 x_1 x_3 x_4 \cdot 10^{-5} - \\&5.62199 x_2 x_3 x_4 \cdot 10^{-5} + 5 \cos (2 \pi \frac{x_5}{100})-2 \end{aligned} \end{aligned}$$
(65)

The constraints \(g_1(\cdot ),\dots ,g_4(\cdot )\) are defined are defined as follows:

$$\begin{aligned} g_1(x_1,x_2,z_1,z_2)= - (x_1-50)^2 - (x_2-50)^2 + (20+c_1*c_2)^2 \end{aligned}$$
(66)

where \(c_1\) and \(c_2\) are defined as a function of \(z_1\) and \(z_2\) according to the relations defined in Table 12.

Table 12 Characterization of the variable-dimension search space Goldstein function constraint
$$\begin{aligned} g_2(x_1,x_2,z_2) = - (x_1-50)^2 - (x_2-50)^2 + (20+c_1*c_2)^2 \end{aligned}$$
(67)

where \(c_1 = 0.5\) and \(c_2\) is defined as a function of \(z_2\) according to the relations defined in Table 13.

Table 13 Characterization of the variable-dimension search space Goldstein function constraint
$$\begin{aligned} g_3(x_1,x_2,z_1) = - (x_1-50)^2 - (x_2-50)^2 + (20+c_1*c_2)^2 \end{aligned}$$
(68)

where \(c_2 = 0.7\) and \(c_1\) is defined as a function of \(z_1\) according to the relations defined in Table 14.

Table 14 Characterization of the variable-dimension search space Goldstein function constraint
$$\begin{aligned} g_4(x_1,x_2,z_3,z_4) = - (x_1-50)^2 - (x_2-50)^2 + (20+c_1*c_2)^2 \end{aligned}$$
(69)

where \(c_1\) and \(c_2\) are defined as a function of \(z_3\) and \(z_4\) according to the relations defined in Table 15.

Table 15 Characterization of the variable-dimension search space Goldstein function constraint

Appendix C: Variable-size design space Rosenbrock function

The variable-size design space variant of the Rosenbrock function which is considered for the testing discussed in Sect. 5 is characterized by a global design space with 8 continuous design variables, 3 discrete design variables and 2 dimensional design variables. Depending on the dimensional variable values, 4 different sub-problems can be identified, with total dimensions of 6 to 9, ranging from 4 continuous variables and 2 discrete variables to 6 continuous variables and 3 discrete variables. One of the 2 constraints is only active for half of the sub-problems

The resulting optimization problem can be defined as follows:

$$\begin{aligned} f(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) = {\left\{ \begin{array}{ll} f_1(x_1,x_2,x_3,x_4,z_1,z_2) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 0\\ f_2(x_1,x_2,x_5,x_6,z_1,z_2,z_3) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 1\\ f_3(x_1,x_2,x_3, x_4,x_7, x_8,z_1,z_2) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 0\\ f_4(x_1,x_2,x_5,x_6,x_7, x_8,z_1,z_2,z_3) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 1 \end{array}\right. } \end{aligned}$$
(70)

and:

$$\begin{aligned} g_1(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) = {\left\{ \begin{array}{ll} g_{1_1}(x_1,x_2,x_3,x_4) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 0\\ g_{1_2}(x_1,x_2,x_5,x_6) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 1\\ g_{1_3}(x_1,x_2,x_3, x_4,x_7, x_8) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 0\\ g_{1_4}(x_1,x_2,x_5,x_6,x_7, x_8) \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 1 \end{array}\right. } \end{aligned}$$
(71)

and:

$$\begin{aligned} g_2(\mathbf{x} ,\mathbf{z} ,\mathbf{w} ) = {\left\{ \begin{array}{ll} g_{2_1}(x_1,x_2,x_3,x_4) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 0\\ g_{2_2}(x_1,x_2,x_3, x_4,x_7, x_8) \qquad \text{ if } w_1 = 0 \text{ and } w_2 = 1\\ \text{ Not } \text{ active } \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 0\\ \text{ Not } \text{ active } \qquad \text{ if } w_1 = 1 \text{ and } w_2 = 1 \end{array}\right. } \end{aligned}$$
(72)

The objective functions \(f_1(\cdot ),\dots ,f_4(\cdot )\) as well as the constraints \(g_{1_1}(\cdot ),\dots ,g_{1_4}(\cdot )\) and \(g_{2_1}(\cdot ),\dots ,g_{2_4}(\cdot )\) are defined as follows:

$$\begin{aligned} f_1(x_1,x_2,x_3,x_4,z_1,z_2) = {\left\{ \begin{array}{ll} 100z_0 + \sum _i a_1*a_2(x_{i+1} - x_i)^2+(a_1+a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 0 \\ 100z_0 + \sum _i 0.7a_1*a_2(x_{i+1} - x_i)^2+(a_1-a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 1 \end{array}\right. } \end{aligned}$$
(73)

with \(a_1 = 7\) and \(a_2 = 9\)

$$\begin{aligned} f_2(x_1,x_2,x_5,x_6,z_1,z_2,z_3) = {\left\{ \begin{array}{ll} 100z_0 - 35z_3 + \sum _i a_1*a_2(x_{i+1} - x_i)^2+(a_1+a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 0 \\ 100z_0 - 35z_3 + \sum _i 0.7a_1*a_2(x_{i+1} - x_i)^2+(a_1-a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 1 \end{array}\right. } \end{aligned}$$
(74)

with \(a_1 = 7\) and \(a_2 = 6\)

$$\begin{aligned} f_3(x_1,x_2,x_3,x_4,x_7,x_8,z_1,z_2) = {\left\{ \begin{array}{ll} 100z_0 + \sum _i a_1*a_2(x_{i+1} - x_i)^2+(a_1+a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 0 \\ 100z_0 + \sum _i 0.7a_1*a_2(x_{i+1} - x_i)^2+(a_1-a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 1 \end{array}\right. } \end{aligned}$$
(75)

with \(a_1 = 10\) and \(a_2 = 9\)

$$\begin{aligned} f_2(x_1,x_2,x_5,x_6,x_7,x_8,z_1,z_2,z_3) = {\left\{ \begin{array}{ll} 100z_0 - 35z_3 + \sum _i a_1*a_2(x_{i+1} - x_i)^2+(a_1+a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 0 \\ 100z_0 - 35z_3 + \sum _i 0.7a_1*a_2(x_{i+1} - x_i)^2+(a_1-a_2)/10(1-x_i)^2 \quad \text{ if } z_2 = 1 \end{array}\right. } \end{aligned}$$
(76)

with \(a_1 = 10\) and \(a_2 = 6\)

$$\begin{aligned} g_{1_1}(x_1,x_2,x_3,x_4)&= \sum _i -(x_i-1)^3 + x_{i+1} - 2.6 \end{aligned}$$
(77)
$$\begin{aligned} g_{1_2}(x_1,x_2,x_5,x_6)&= \sum _i -(x_i-1)^3 + x_{i+1} - 2.6 \end{aligned}$$
(78)
$$\begin{aligned} g_{1_3}(x_1,x_2,x_3,x_4,x_7,x_8)&= \sum _i -(x_i-1)^3 + x_{i+1} - 2.6 \end{aligned}$$
(79)
$$\begin{aligned} g_{1_4}(x_1,x_2,x_5,x_6,x_7,x_8)&= \sum _i -(x_i-1)^3 + x_{i+1} - 2.6 \end{aligned}$$
(80)
$$\begin{aligned} g_{2_1}(x_1,x_2,x_3,x_4)&= \sum _i -x_i - x_{i+1} + 0.4 \end{aligned}$$
(81)
$$\begin{aligned} g_{2_2}(x_1,x_2,x_5,x_6)&= \sum _i -x_i - x_{i+1} + 0.4 \end{aligned}$$
(82)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pelamatti, J., Brevault, L., Balesdent, M. et al. Bayesian optimization of variable-size design space problems. Optim Eng 22, 387–447 (2021). https://doi.org/10.1007/s11081-020-09520-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-020-09520-z

Keywords

  • Variable-size design space optimization problems
  • Mixed-variable optimization problems
  • Bayesian optimization
  • Discrete variables