A solution method for mixed-variable constrained blackbox optimization problems

Abstract

Many real-world application problems encountered in industry have no analytical formulation, that is they are blackbox optimization problems, and often make use of expensive numerical simulations. We propose a new blackbox optimization algorithm named BOA to solve mixed-variable constrained blackbox optimization problems where the evaluations of the blackbox functions are computationally expensive. The algorithm is two-phased: in the first phase it looks for a feasible solution and in the second phase it tries to find other feasible solutions with better objective values. Our implementation of the algorithm constructs surrogates approximating the blackbox functions and defines subproblems based on these models. The open-source blackbox optimization solver NOMAD is used for the resolution of the subproblems. Experiments performed on instances stemming from the literature and two automotive applications encountered at Stellantis show promising results of BOA in particular with cubic RBF models. The latter generally outperforms two surrogate-assisted NOMAD variants on the considered problems.

Appendix: Formulations of the test problems

1.1 A.1 Problem C1 (Floudas and Pardalos 1990)

This is the well-known G01 benchmark problem.

$$\begin{aligned} \left\{ \begin{array}{ll} \min 5 \sum _{i=1}^4{x_i} - 5 \sum _{i=1}^4{x_i^2} - \sum _{i=5}^{13}{x_i}\\ \text{ subject } \text{ to }\\ 2 x_1 + 2 x_2 + x_{10} + x_{11} - 10 \le 0 \\ 2 x_1 + 2 x_3 + x_{10} + x_{12} - 10\le 0 \\ 2 x_2 + 2 x_3 + x_{11} + x_{12} - 10\le 0 \\ -8 x_1 + x_{10}\le 0 \\ -8 x_2 + x_{11}\le 0 \\ -8 x_3 + x_{12}\le 0 \\ -2 x_4 - x_5 + x_{10}\le 0 \\ -2 x_6 - x_7 + x_{11} \le 0 \\ -2 x_8 - x_9 + x_{12} \le 0 \\ x_i\in [0,1], \ i=1,2,\ldots ,9,13\\ x_i\in [0,100], \ i=10,11,12. \end{array} \right. \end{aligned}$$

1.2 A.2 Problem C2 (Hock and Schittkowski 1980)

This is the well-known G07 benchmark problem.

$$\begin{aligned} \left\{ \begin{array}{ll} \min x_1^2+ x_2^2+ x_1x_2 - 14x_1 - 16x_2 + \left( x_3 - 10\right) ^2 + 4\left( x_4 - 5\right) ^2 + \left( x_5 - 3\right) ^2\\ \quad \quad +2\left( x_6 - 1\right) ^2 \\ \quad \quad + 5x_7^2 + 7\left( x_8 - 11\right) ^2 + 2\left( x_9 - 10\right) ^2 + \left( x_{10} -7\right) ^2 + 45\\ {\text{ subject } \text{ to }}\\ \quad \quad -105 + 4x_1 + 5x_2 - 3x_7 + 9x_8 \le 0\\ \quad \quad 10x_1 - 8x_2 -17x_7 + 2x_8 \le 0\\ \quad \quad -8x_1 + 2x_2 + 5x_9 - 2x_{10} - 12 \le 0\\ \quad \quad 3\left( x_1 - 2\right) ^2 +4\left( x_2 - 3\right) ^2 + 2x_3^2 - 7x_4 - 120 \le 0\\ \quad \quad 5 x_1^2 + 8x_2 + \left( x_3 - 6\right) ^2 - 2x_4 - 40 \le 0\\ \quad \quad x_1^2 + 2\left( x_2 - 2\right) ^2 - 2x_1 x_2 + 14x_5 - 6x_6 \le 0\\ \quad \quad 0.5\left( x_1 - 8\right) ^2 + 2\left( x_2 - 4\right) ^2 + 3x_5^2 - x_6 - 30 \le 0\\ \quad \quad -3x_1 + 6x_2 + 12\left( x_9 - 8\right) ^2 - 7x_{10} \le 0\\ {x_i\in [-10,10], \ i=1,2,\ldots ,10.} \end{array} \right. \end{aligned}$$

1.3 A.3 Problem C3 (Himmelblau 1972)

This is the well-known G19 benchmark problem.

$$\begin{aligned}{} & {} a= \begin{bmatrix} 16&{} 2&{} 0&{} 1&{} 0\\ 0&{} -2&{} 0&{} 0.4&{} 2\\ -3.5&{} 0&{} 2&{} 0&{} 0\\ 0&{} -2&{} 0&{} -4&{} -1\\ 0&{} -9&{} -2&{} 1&{} -2.8\\ 2&{} 0&{} -4&{} 0&{} 0\\ -1&{} -1&{} -1&{} -1&{} -1\\ -1&{} -2&{} -3&{} -2&{} -1\\ 1&{} 2&{} 3&{} 4&{} 5\\ 1&{} 1&{} 1&{} 1&{} 1 \end{bmatrix} b= \begin{bmatrix} -40\\ -2\\ -0.25\\ -4\\ -4\\ -1\\ -40\\ -60\\ 5\\ 1 \end{bmatrix}\\{} & {} c= \begin{bmatrix} 30&{} -20&{} -10&{} 32&{} -10\\ -20&{} 39&{} -6&{} -31&{} 32\\ -10&{} -6&{} 10&{} -6&{} -10\\ 32&{} -31&{} -6&{} 39&{} -20\\ -10&{} 32&{} -10&{} -20&{} 30\\ \end{bmatrix} d = \begin{bmatrix} 4\\ 8\\ 10\\ 6\\ 2 \end{bmatrix} e = \begin{bmatrix} -15\\ -27\\ -36\\ -18\\ -12 \end{bmatrix} \\{} & {} \left\{ \begin{array}{ll} \min \sum _{j=1}^5{\sum _{i=1}^5}{c_{ij}x_{(10+i)} x_{(10+j)}} +2\sum _{i=1}^5{d_i x_{(10+i)}^3}-\sum _{i=1}^{10}{b_ix_i}\\ {\text{ subject } \text{ to }}\\ {-2\sum _{i=1}^{5}{c_{ij} x_{(10+i)} - 3d_j x_{(10+j)}^2 - e_j+\sum _{i=1}^{10}{a_{ij}x_i}} \le 0, \ j=1,2,\ldots ,5}\\ {x_i\in [0,10], \ i=1,2,\ldots ,15.} \end{array} \right. \end{aligned}$$

1.4 A.4 Problem C4 (Paul 1987; Pant et al. 2009; Kumar et al. 2020): optimal design of an industrial refrigeration system

A design problem expressed as a non-linear inequality constrained optimization problem.

$$\begin{aligned} \left\{ \begin{array}{ll} \min 63098.88 x_2x_4x_{12}+5441.5 x_2^2x_{12}+115055.5 x_{2}^{1.664}x_6+\\ \quad \quad {6172.27x_2^2x_6+63098.88x_1x_3x_{11}+5441.5x_1^2x_{11} + 115055.5 x_1^{1.664}x_5} \\ \quad \quad {+6172.27 x_1^2x_5+140.53x_1x_{11} +281.29x_3x_{11}+70.26x_1^2+281.29x_1x_3}\\ \quad \quad {+281.29x_3^2+ 14437x_8^{1.8812}x_{12}^{0.3424}x_{10}x_{14}^{-1}x_1^2x_7x_9^{-1} +20470.2 x_7^{2.893}x_{11}^{0.316}x_1^2}\\ {\text{ subject } \text{ to }}\\ {1.524 x_7^{-1}-1 \le 0}\\ {1.524 x_8^{-1}-1\le 0}\\ {0.07789 x_1 -2x_7^{-1}x_9-1\le 0}\\ {7.05305 x_9^{-1} x_1^2 x_{10} x_{8}^{-1}x_2^{-1}x_{14}^{-1} -1 \le 0}\\ {0.0833 x_{13}^{-1}x_{14}-1\le 0}\\ {47.136 x_{2}^{0.333} x_{10}^{-1} x_{12} -1.333 x_8 x_{13}^{2.1195}+62.08 x_{13}^{2.1195}x_{12}^{-1}x_8^{0.2}x_{10}^{-1}-1 \le 0}\\ {0.04771 x_{10} x_8^{1.8812} x_{12}^{0.3424}-1\le 0}\\ {0.0488 x_9 x_7^{1.893} x_{11}^{0.316}-1 \le 0}\\ {0.0099 x_1 x_3^{-1}-1\le 0}\\ {0.0193 x_2 x_4^{-1}-1\le 0}\\ {0.0298 x_1 x_5^{-1}-1 \le 0}\\ {0.056 x_2 x_{6}^{-1}-1\le 0}\\ {2x_{9}^{-1}-1\le 0}\\ {2x_{10}^{-1}-1\le 0}\\ {x_{12}x_{11}^{-1}-1\le 0}\\ {x_i\in [0.001,5], \ i=1,2,\ldots ,14.} \end{array} \right. \end{aligned}$$

1.5 A.5 Problem C5 (Grandhi and Venkayya 1988; Kumar et al. 2020): 10-bar truss design

The aim is to minimize the weight of a truss structure subject to frequency constraints. The truss is represented as a finite element structure that has 10 two-dimensional bar elements and 6 nodes.

$$\begin{aligned} \left\{ \begin{array}{ll} \min f\left( {\overline{x}}\right) =\sum _{i=1}^{10} L_i \rho A_i \\ {\text{ subject } \text{ to }}\\ {\frac{7}{\omega _1\left( {\overline{x}}\right) } -1\le 0}\\ {\frac{15}{\omega _2\left( {\overline{x}}\right) }-1 \le 0}\\ {\frac{20}{\omega _3\left( {\overline{x}}\right) }-1 \le 0} \end{array} \right. \end{aligned}$$

with bounds:

$$\begin{aligned} A_i\in [6.45\cdot 10^{-5}\,\ 5\cdot 10^{-3}], \ i=1,2,\ldots ,10, \end{aligned}$$

where

$$\begin{aligned}{} & {} {\overline{x}}=\left\{ A_1,A_2,\ldots ,A_{10}\right\} ,\ \rho =2770, \\{} & {} L_i = \left\{ \begin{array}{ll} 9.144 &{}\text { if } i \le 6\\ 9.144 \cdot \sqrt{2} &{}\text { otherwise.} \end{array} \right. \end{aligned}$$

The functions $\omega _1\left( {\overline{x}}\right) , \omega _2\left( {\overline{x}}\right)$ and $\omega _3\left( {\overline{x}}\right)$ are computed from matrices K and M, that need to be assembled from smaller matrices, and their lowest eigenvalues.

Let

$$\begin{aligned} M^{(i)} = \tfrac{1}{6} \rho L_i A_i \begin{bmatrix} 2 &{} 0 &{} 1 &{} 0\\ 0 &{} 2 &{} 0 &{} 1\\ 1 &{} 0 &{} 2 &{} 0\\ 0 &{} 1 &{} 0 &{} 2 \end{bmatrix} \text { and } K^{(i)} = \tfrac{E A_i}{L_i^3} \begin{bmatrix} -l_i m_i &{} -m_i^2 &{} l_i m_i &{} m_i^2\\ -l_i^2 &{} -l_i m_i &{} l_i^2 &{} l_i m_i\\ l_i m_i &{} m_i^2 &{} -l_i m_i &{} -m_i^2\\ l_i^2 &{} l_i m_i &{} -l_i^2 &{} -l_i m_i \end{bmatrix} \end{aligned}$$

with

$$\begin{aligned} E = 6.98 \cdot 10^{10}, l_i = \left\{ \begin{array}{ll} 0 &{}\text { if } i \in \{5,6\}\\ 9.144 &{}\text { otherwise} \end{array} \right. \text { and } m_i = \left\{ \begin{array}{ll} 0 &{}\text { if } i \le 4\\ -9.144 &{} \text { if } i \in \{7,9\}\\ 9.144 &{}\text { otherwise.} \end{array} \right. \end{aligned}$$

Let ${\mathcal {I}} = \begin{bmatrix} 5 &{} 6 &{} 9 &{} 10\\ 1 &{} 2 &{} 5 &{} 6\\ 7 &{} 8 &{} 11 &{} 12\\ 3 &{} 4 &{} 7 &{} 8\\ 5 &{} 6 &{} 7 &{} 8\\ 1 &{} 2 &{} 3 &{} 4\\ 7 &{} 8 &{} 9 &{} 10\\ 5 &{} 6 &{} 11 &{} 12\\ 3 &{} 4 &{} 5 &{} 6\\ 1 &{} 2 &{} 7 &{} 8 \end{bmatrix},$ we denote ${\mathcal {I}}_{i,:} = \begin{bmatrix} {\mathcal {I}}_{i,1}&{\mathcal {I}}_{i,2}&{\mathcal {I}}_{i,3}&{\mathcal {I}}_{i,4} \end{bmatrix}$, the $i^{\text {th}}$ line of ${\mathcal {I}}$ where, for all $j \in \{1,2,3,4\}$, ${\mathcal {I}}_{i,j}$ is the element of the $i^{\text {th}}$ line and $j^{\text {th}}$ column of ${\mathcal {I}}$.

Let $A \in {\mathbb {R}}^{12\times 12}$ be a real square matrix, and $v = [a \ b \ c \ d]$ be a line vector with $\{a, b, c, d\} \in \{1,2,\ldots , 12\}^{4}$, we denote $A[v] = A[a \ b \ c \ d] = \begin{bmatrix} A_{aa} &{} A_{ab} &{} A_{ac} &{} A_{ad}\\ A_{ba} &{} A_{bb} &{} A_{bc} &{} A_{bd}\\ A_{ca} &{} A_{cb} &{} A_{cc} &{} A_{cd}\\ A_{da} &{} A_{db} &{} A_{dc} &{} A_{dd} \end{bmatrix}.$

The following procedure describes how $\omega _1\left( {\overline{x}}\right) , \omega _2\left( {\overline{x}}\right)$ and $\omega _3\left( {\overline{x}}\right)$ are computed:

Algorithm 5

Compute $\omega _1, \omega _2$ and $\omega _3$

1.6 A.6 Problem C6 (Wang et al. 2018; Kumar et al. 2020): wind farm layout problem

The objective is to minimize the opposite sum of the expected power output of each wind turbine i with minimum distance constraints between the wind turbines. The optimization problem is as follows:

$$\begin{aligned} \left\{ \begin{array}{ll} \min {-\sum _{i=1}^{N}{E\left( P_i\right) }}\\ {\text{ subject } \text{ to }}\\ {5R - \sqrt{\left( x_i-x_j\right) ^2 + \left( y_i-y_j\right) ^2} \le 0, \ j=1,2,\ldots ,N \text{ and } j\ne i}\\ \end{array} \right. \end{aligned}$$

where

• ${\underline{x}}+R \le x_i \le {\overline{x}}-R$ and ${\underline{y}}+R \le y_i \le {\overline{y}}-R, \forall i=1,2,\ldots ,N$, with $N = 15$,

• ${\underline{x}} = [0 \ 0 \ldots 0]^\top$ and ${\underline{y}} = [0 \ 0 \ldots 0]^\top$ are lower bounds for all components of x and y, respectively,

• ${\overline{x}} = [2000 \ 2000 \ldots 2000]^\top$ and ${\overline{y}} = [2000 \ 2000 \ldots 2000]^\top$ are upper bounds for all components of x and y respectively.

  $$\begin{aligned} \begin{array}{ll} E\left( P_i\right) = \sum _{n=1}^{h}{\xi _n \left\{ P_r\left( e^{-\left( \nu _{r}/{c'_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }\right) ^{k_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }} -e^{-\left( \nu _{co}/c'_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) \right) ^{k_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }} \right) \right. }\\ \quad \left. + \sum _{j=1}^s{ \left( e^{-\left( \nu _{j-1}/{c'_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }\right) ^{k_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }} -e^{-\left( \nu _{j}/c'_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) \right) ^{k_i\left( \left( \theta _{n-1}+\theta _n\right) /2\right) }} \right) }\right. \\ \quad \left. \frac{e^{\left( \nu _{j-1}+\nu _j\right) /2}}{\alpha +\beta e^{\left( \nu _{j-1}+\nu _j\right) /2}} \right\} \end{array} \end{aligned}$$

• $\xi _n$ is the frequency of the interval $\left[ \theta _{n-1},\theta _n\right)$.

The following parameters are set: $h = 24$, $s = 36$, $R = 40$, $P_r = 1500$, $\alpha = 6.0268$, $\beta = 0.0007$, $\nu _{r} = 14$, $\nu _{co} = 25$ and $\nu _{ci} = 3.5$.

$\forall n \in \{1,2,\ldots ,h\}, \theta _n = \theta _{n-1} + \tfrac{360}{h}$ with $\theta _0 = 0^{\circ }$.

$\forall j \in \{1,2,\ldots ,s\}, \nu _j = \nu _{j-1} + \tfrac{(\nu _r - \nu _{ci})}{s}$ with $\nu _0 = \nu _{ci}$.

For all $n \in \{1,2,\ldots ,h\}$, we denote $\theta ^{(n)} = \tfrac{\theta _{n-1}+\theta _n}{2}$.

For all $n \in \{1,2,\ldots ,h\}$, $k_i(\theta ^{(n)}) = 2$ and the following table gives the values of $c_i(\theta ^{(n)})$ and $\chi _n$ for each n.

n	$c_i(\theta ^{(n)})$	$\chi _n$	n	$c_i(\theta ^{(n)})$	$\chi _n$	n	$c_i(\theta ^{(n)})$	$\chi _n$
1	7	0.0003	9	7	0.0626	17	4.6	0.0041
2	5	0.0072	10	7	0.0802	18	2.6	0.0008
3	5	0.0237	11	8	0.1025	19	8	0.001
4	5	0.0242	12	9.5	0.1445	20	5	0.0005
5	5	0.0222	13	10	0.1909	21	6.4	0.0013
6	4	0.0301	14	8.5	0.1162	22	5.2	0.0031
7	5	0.0397	15	8.5	0.0793	23	4.5	0.0085
8	6	0.0268	16	6.5	0.0082	24	3.9	0.0222

Moreover, $c'_i(\theta ^{(n)}) = c_i(\theta ^{(n)}) (1 - VD_i)$,

where $VD_i = 2 a \sqrt{\sum \limits _{j=1,j \ne i}^{N}{\tfrac{1}{\left( 1+ \tfrac{\kappa }{R}|(x_j - x_i)cos(\theta ^{(n)}) + (y_j - y_i)sin(\theta ^{(n)})|\right) ^4}}}$  

$a = 0.5 \cdot (1 - \sqrt{1 - C_T})$, $C_T = 0.8$ and $\kappa = 0.01$.

1.7 A.7 Problem I1 (Floudas and Pardalos 1990; Müller et al. 2014)

This is the problem G01 with integrality constraints on the variables.

$$\begin{aligned} \left\{ \begin{array}{ll} \min {5\sum _{i=1}^{4}{x_i}-5\sum _{i=1}^{4}{x_i^2}-\sum _{i=5}^{13}{x_i}}\\ \text{ subject } \text{ to }\\ {2x_1+2x_2+x_{10}+x_{11} - 10 \le 0}\\ {2x_1+2x_3+x_{10}+x_{12} - 10 \le 0}\\ {2x_2+2x_3+x_{11}+x_{12} - 10 \le 0}\\ {-8x_1+x_{10}\le 0}\\ {-8x_2+x_{11}\le 0}\\ {-8x_3+x_{12}\le 0}\\ {-2x_4-x_5+x_{10}\le 0}\\ {-2x_6-x_7+x_{11}\le 0}\\ {-2x_8-x_9+x_{12}\le 0}\\ {x_i\in \{0,1\}, \ i=1,2,\ldots ,9,13}\\ {x_i\in \{0,1,\ldots ,100\}, \ i=10,11,12}\\ \end{array} \right. \end{aligned}$$

1.8 A.8 Problem I2 (Bussieck et al. 2003; Müller et al. 2014): hmittelman

The binary nonlinear problem hmittelman.

$$\begin{aligned} \left\{ \begin{array}{ll} \min {10y_1+7y_2+y_3+12y_4+8y_5+3y_6+y_7+5y_8+3y_9}\\ \text{ subject } \text{ to }\\ {3y_1-12y_2-8y_3+y_4-7y_9+2y_{10} +2 \le 0}\\ {y_2-10y_3-5y_5+y_6+7y_7+y_8 + 1 \le 0 }\\ {5y_1-3y_2-y_3-2y_8+y_{10} + 1 \le 0}\\ {3y_2-5y_1+y_3+2y_8-y_{10} - 1 \le 0}\\ {-4y_3-2y_4-5y_6+y_7-9y_8-2y_9 + 3 \le 0}\\ {9y_2-12y_4-7y_5+6y_6+2y_8-15y_9+3y_{10} + 7 \le 0}\\ {5y_2-8y_1+2y_3-7y_4-y_5-5y_7-10y_9 + 1 \le 0}\\ {y_1=x_5x_7x_9x_{10}x_{14}x_{15}x_{16}}\\ {y_2=x_1x_2x_3x_4x_8x_{11}}\\ {y_3=x_3x_4x_6x_7x_8}\\ {y_4=x_3x_4x_8x_{11}}\\ {y_5=x_6x_7x_8x_{12}}\\ {y_6=x_6x_7x_9x_{14}x_{16}}\\ {y_7=x_9x_{10}x_{14}x_{16}}\\ {y_8=x_5x_{10}x_{14}x_{15}x_{16}}\\ {y_9=x_1x_2x_{11}x_{12}}\\ {y_{10}=x_{13}x_{14}x_{15}x_{16}}\\ {x_i\in \{0,1\}, \ i=1,2,\ldots ,16.} \end{array} \right. \end{aligned}$$

1.9 A.9 Problem I3 (Floudas and Pardalos 1990; Müller et al. 2014)

The G01 problem with integrality constraints and modified bounds.

$$\begin{aligned} \left\{ \begin{array}{ll} \min {5\sum _{i=1}^{4}{x_i}-5\sum _{i=1}^{4}{x_i^2}-\sum _{i=5}^{13}{x_i}}\\ \text{ subject } \text{ to }\\ {2x_1+2x_2+x_{10}+x_{11} - 10 \le 0}\\ {2x_1+2x_3+x_{10}+x_{12} - 10 \le 0}\\ {2x_2+2x_3+x_{11}+x_{12} - 10 \le 0}\\ {-8x_1+x_{10}\le 0}\\ {-8x_2+x_{11}\le 0}\\ {-8x_3+x_{12}\le 0}\\ {-2x_4-x_5+x_{10}\le 0}\\ {-2x_6-x_7+x_{11}\le 0}\\ {-2x_8-x_9+x_{12}\le 0}\\ {x_i\in \{0,1,\ldots ,100\}, \ i=1,2,\ldots ,13.} \end{array} \right. \end{aligned}$$

1.10 A.10 Problem MI1 (Berman and Ashrafi 1993; Müller et al. 2013)

This is a modification of a reliability problem.

$$\begin{aligned} \left\{ \begin{array}{ll} \min {-x_5x_6x_7}\\ \text{ subject } \text{ to }\\ {-x_8-x_9-x_{10}+1\le 0}\\ {-x_1-x_2-x_{11}+1\le 0}\\ {-x_3-x_4+1\le 0}\\ {2x_1+x_2+3x_3+2x_4+3x_8+x_9+2x_{10}+3x_{11}-10\le 0}\\ {-\log (1-x_5)+\log (0.1)x_8+\log (0.2)x_9+\log (0.15)x_{10}\le 0}\\ {\log (0.2)x_1+\log (0.15)x_2-\log \left( 1-x_6\right) +\log (0.05)x_{11}\le 0}\\ {\log (0.02)x_3+\log (0.06)x_4-\log \left( 1-x_7\right) \le 0}\\ {x_i\in [0,1], \ i=1,2,3,4}\\ {x_5\in [0,0.997],\ x_6\in [0,0.9985],\ x_7\in [0,0.9988]}\\ {x_i\in \{0,1\}\, i=8,9,10,11.}\\ \end{array} \right. \end{aligned}$$

1.11 A.11 Problem MI2 (Yuan et al. 1988; Müller et al. 2013)

This is a purely mathematical problem with linear and nonlinear constraints.

$$\begin{aligned} \left\{ \begin{array}{ll} \min (x_1-1)^2 + (x_2-2)^2 + (x_3-1)^2 - \log (x_{4}+1) + (x_{5}-1)^2 \\ \quad + (x_{6}-2)^2 + (x_{7}-3)^2\\ \text{ subject } \text{ to }\\ x_8 + x_9 + x_{10} + x_{5} + x_{6} + x_{7} - 5 \le 0\\ x_3^2 + x_5^2 + x_6^2 + x_7^2 - 5.5 \le 0\\ x_8 + x_{5} - 1.2 \le 0\\ x_9 + x_{6} - 1.8 \le 0\\ x_{10} + x_{7} - 2.5\le 0\\ x_{11} + x_{5} - 1.2\le 0\\ x_2^2 + x_6^2 - 1.64\le 0\\ x_3^2 + x_7^2 - 4.25\le 0\\ x_2^2 + x_7^2 - 4.64\le 0\\ x_1 - x_8\le 0\\ x_2 - x_9\le 0\\ x_3 - x_{10}\le 0\\ x_{4} - x_{11}\le 0\\ x_i \in [0,1], \ i=1,...,4\\ x_i \in [0,10], \ i=5,...,7\\ x_i \in \{ 0,1 \}, \ i=8,...,11 \end{array} \right. \end{aligned}$$

1.12 A.12 Problem MI3 (Kuo et al. 2001; Müller et al. 2013): bridge system

It is a reliability-redundancy allocation problem on a bridge network.

$$\begin{aligned} \left\{ \begin{array}{ll} \min - R_1R_2 - R_3R_4 - R_1R_4R_5 - R_2R_3R_5 + R_1R_2R_3R_4 + R_1R_2R_3R_5 \\ + R_1R_2R_4R_5 + R_1R_3R_4R_5 + R_2R_3R_4R_5 - 2 R_1R_2R_3R_4R_5\\ \text{ subject } \text{ to }\\ \sum _{i=1}^5 p_iu_i^2 - 110 \le 0\\ \sum _{i=1}^5 (\alpha _i(\frac{-1000}{\log (x_i)})^{1.5})(u_i+e^{\frac{u_i}{4}}) - 175 \le 0\\ \sum _{i=1}^5\omega _iu_ie^{\frac{u_i}{4}} - 200 \le 0, \end{array} \right. \end{aligned}$$

where $R_i = 1- (1-x_i)^{u_i}$, $0 \le x_i \le 1-10^{-6}$, and $u_i \in \{1,2,\ldots ,10\}, \forall i=1,2,\ldots ,5$. Besides, $\alpha = (2.330, 1.450, 0.541, 8.050, 1.950)\cdot 10^{-5}$, $p = (1, 2, 3, 4, 2)$ and $\omega = (7, 8, 8, 6, 9)$.

1.13 A.13 Problem MI4 (Kuo et al. 2001; Müller et al. 2013): series–parallel system

This is a reliability-redundancy allocation problem on a series–parallel system.

$$\begin{aligned} \left\{ \begin{array}{ll} \min - 1 + (1 - R_1R_2)(1 - (1-R_3)(1-R_4)R_5))\\ \text{ subject } \text{ to }\\ \sum _{i=1}^5 p_i u_i^2 - 180 \le 0\\ \sum _{i=1}^5 (\alpha _i(\frac{-1000}{\log (x_i)})^{1.5})(u_i+e^{\frac{u_i}{4}}) - 175 \le 0\\ \sum _{i=1}^5\omega _i u_i e^{\frac{u_i}{4}} - 100 \le 0, \end{array} \right. \end{aligned}$$

where $R_i = 1- (1-x_i)^{u_i}$, $0 \le x_i \le 1-10^{-6}$, and $u_i \in \{1,2,\ldots ,10\}, \forall i=1,2,\ldots ,5$. Parameters: $\alpha = (2.5, 1.45, 0.541, 0.541, 2.1)\cdot 10^{-5}$, $p = (2, 4, 5, 8, 4)$ and $\omega = (3.5, 4, 4, 3.5, 4.5)$.

1.14 A.14 Problem MI5 (Grossmann and Sargent 1979; Kumar et al. 2020): multi-product batch plant

This problem aims at designing a multi-product batch plant with 3 serial batch processing stages manufacturing 2 different products.

$$\begin{aligned} \left\{ \begin{array}{ll} \min 250 \sum _{j=1}^3 N_j V_j^{0.6} \\ \text{ subject } \text{ to }\\ 2B_1+4B_2-V_1 \le 0 \\ 3B_1+6B_2-V_2\le 0\\ 4B_1+3B_2-V_3\le 0\\ \frac{40000T_{L1}}{B_1}+\frac{20000T_{L2}}{B_2}-6000\le 0\\ 8-N_1T_{L1}\le 0\\ 20-N_2T_{L1}\le 0\\ 8-N_3T_{L1}\le 0\\ 16-N_1T_{L2}\le 0\\ 4-N_2T_{L2}\le 0\\ 4-N_3T_{L2}\le 0, \end{array} \right. \end{aligned}$$

where $1 \le N_1,N_2,N_3 \le 3$, $250\le V_1,V_2,V_3 \le 2500$, $\frac{20}{3} \le T_{L1}, T_{L2} \le 20$, $\frac{20}{3}T_{L1} \le B_1 \le 625$ and $\frac{10}{3}T_{L2} \le B_2 \le \frac{1250}{3}$.

1.15 A.15 Problem MI6 (Gupta et al. 2007; Kumar et al. 2020): rolling-element bearing

This problem aims at optimizing the internal geometry of a rolling bearing.

$$\begin{aligned} \left\{ \begin{array}{ll} \min {\left\{ \begin{array}{ll} f_cz^{2/3}d_b^{1.8} &{} \text {if } d_b\le 25.4 \\ 3.647f_cz^{2/3}d_b^{1.4} &{} \text{ otherwise } \\ \end{array}\right. } \\ \text{ subject } \text{ to }\\ z-\frac{\Phi _0}{2\arcsin (d_b/d_m)} -1 \le 0\\ k_{Dmin}(D-d)-2d_b \le 0\\ 2d_b - k_{Dmax}(D-d) \le 0 \\ \zeta B_w - d_b \le 0 \\ 0.5(D+d) - d_m \le 0 \\ d_m - (0.5+e)(D+d) \le 0 \\ \epsilon d_b - 0.5(D-d_m - d_b) \le 0 \\ 0.515 - f_i \le 0 \\ 0.515-f_0 \le 0, \end{array} \right. \end{aligned}$$

where $0.5(D+d) \le d_m \le 0.6(D+d)$, $0.15(D-d)\le d_b \le 0.45(D-d)$, $4\le z \le 50$, $0.515\le f_i,f_0\le 0.6$, $0.4 \le k_{Dmin} \le 0.5$, $0.6 \le k_{Dmax} \le 0.7$, $0.3 \le \epsilon \le 0.4$, $0.02 \le e \le 0.1$ and $0.6 \le \zeta \le 0.85$.

Besides, $f_c = 37.91(1+(1.04(\frac{1-\gamma }{1+\gamma })^{1.72}(\frac{f_i(2f_0-1)}{f_0(2f_i-1)})^{0.41})^{10/3})^{-0.3}$, $\gamma = \frac{d_b}{d_m}$, $f_i= \frac{r_i}{d_b}$, $f_0 = \frac{r_0}{d_b}$, $\Phi _0 = 2\pi - 2\arccos (\dfrac{(\frac{D-d}{2} -3\frac{T}{4})^2 + (\frac{D}{2} - \frac{T}{4} -d_b)^2 - (\frac{d}{2} + \frac{T}{4})^2}{2(\frac{D-d}{2}-3\frac{T}{4})(\frac{D}{2}-\frac{T}{4}-d_b)})$, $T=D-d-2d_b$, $D=160$, $d=90$ and $B_w= 30$.

1.16 A.16 Problem MV2 (Hock and Schittkowski 1980; Crélot et al. 2017)

This is the well-known G07 problem where some variables are imposed to be discrete.

$$\begin{aligned} \left\{ \begin{array}{ll} \min x_1^2 + x_2^2 + x_1 x_2 - 14 x_1 - 16 x_2 + (x_3 - 10)^2 + 4 (x_4 - 5)^2 + (x_5 - 3)^2\\ + 2 (x_6 - 1)^2 + 5 x_7^2 + 7 (x_8 - 11)^2 + 2 (x_9 - 10)^2 + (x_{10} - 7)^2 + 45\\ \text{ subject } \text{ to }\\ -105 + 4 x_1 + 5 x_2 - 3 x_7 + 9 x_8 \le 0\\ 10 x_1 - 8 x_2 -17 x_7 + 2 x_8 \le 0\\ -8 x_1 + 2 x_2 + 5 x_9 - 2 x_{10} - 12\le 0\\ 3 (x_1 - 2)^2 +4 (x_2 - 3)^2 + 2 x_3^2 - 7 x_4 - 120\le 0\\ 5 x_1^2 + 8 x_2 + (x_3 - 6)^2 - 2 x_4 - 40\le 0\\ x_1^2 + 2 (x_2 - 2)^2 - 2 x_1 x_2 + 14 x_5 - 6 x_6\le 0\\ 0.5 (x_1 - 8)^2 + 2 (x_2 - 4)^2 + 3 x_5^2 - x_6 - 30\le 0\\ -3 x_1 + 6 x_2 + 12 (x_9 - 8)^2 - 7 x_{10}\le 0, \end{array} \right. \end{aligned}$$

where $x_i \in \{-10,-5,0,1.3,2.2,5, 8.2, 8.7, 9.5, 10\}, \forall i=1,2,\ldots ,6$, $-10 \le x_7,x_8 \le 10$ and $x_9,x_{10} \in \{-10,-9,\ldots ,10\}$.

1.17 A.17 Problem MV3 (Gu et al. 2001; Gandomi et al. 2011): car side impact design

This problem uses a simplified regression model of the finite element model of a car side impact. The aim is to minimize the weight of the car subject to nonlinear inequality constraints.

$$\begin{aligned} \left\{ \begin{array}{ll} \min 1.98 + 4.90 x_1 + 6.67 x_2 + 6.98 x_3 + 4.01 x_4 + 1.78 x_5 + 2.73 x_7 \\ \text{ subject } \text{ to }\\ 1.16 - 0.3717 x_2 x_4 - 0.00931 x_2 x_{10} - 0.484 x_3 x_9 + 0.01343 x_6 x_{10} - 1 \le 0\\ 0.261 - 0.0159 x_1 x_2 - 0.188 x_1 x_8 - 0.019 x_2 x_7 + 0.0144 x_3 x_5 + 0.0008757 x_5 x_{10}\\ + 0.08045 x_6 x_9 + 0.00139 x_8 x_{11}\\ + 0.00001575 x_{10} x_{11} - 0.32 \le 0\\ 0.214 + 0.00817 x_5 - 0.131 x_1 x_8 - 0.0704 x_1 x_9 + 0.03099 x_2 x_6 - 0.018 x_2 x_7 + 0.0208 x_3 x_8\\ + 0.121 x_3 x_9 - 0.00364 x_5 x_6 + 0.0007715 x_5 x_{10} - 0.0005354 x_6 x_{10} + 0.00121 x_8 x_{11} - 0.32 \le 0\\ 0.074 - 0.061 x_2 - 0.163 x_3 x_8 + 0.001232 x_3 x_{10} - 0.166 x_7 x_9 + 0.227 x_2^2 - 0.32 \le 0\\ 28.98 + 3.818 x_3 - 4.2 x_1 x_2 + 0.0207 x_5 x_{10} + 6.63 x_6 x_9 - 7.7 x_7 x_8 + 0.32 x_9 x_{10} -32 \le 0\\ 33.86 + 2.95 x_3 + 0.1792 x_{10} \\ - 5.057 x_1 x_2 - 11.0 x_2 x_8 - 0.0215 x_5 x_{10} - 9.98 x_7 x_8 + 22.2 x_8 x_9 - 32 \le 0\\ 46.36 - 9.9 x_2 - 12.9 x_1 x_8 + 0.1107 x_3 x_{10} - 32 \le 0\\ 4.72 - 0.5 x_4 - 0.19 x_2 x_3 - 0.0122 x_4 x_{10} + 0.009325 x_6 x_{10} + 0.000191 x_{11}^2 - 4 \le 0\\ 10.58 - 0.674 x_1 x_2 - 1.95 x_2 x_8 + 0.02054 x_3 x_{10} - 0.0198 x_4 x_{10} + 0.028 x_6 x_{10} - 9.9 \le 0\\ 16.45 - 0.489 x_3 x_7 - 0.843 x_5 x_6 + 0.0432 x_9 x_{10} - 0.0556 x_9 x_{11} - 0.000786 x_{11}^2 - 15.7 \le 0,\\ \end{array} \right. \end{aligned}$$

where $0.5\le x_1,x_3,x_4\le 1.5$, $0.45 \le x_2 \le 1.35$, $0.875\le x_5 \le 2.625$, $0.4 \le x_6,x_7 \le 1.2$ $x_8, x_9 \in \{ 0.192, 0.345\}$, $-30 \le x_{10},x_{11} \le 30$.

1.18 A.18 Problem MV4 (Thanedar and Vanderplaats 1995; Gandomi et al. 2011): stepped cantilever beam design

This problem minimizes the volume of a stepped cantilever beam.

$$\begin{aligned} \left\{ \begin{array}{ll} \min l (x_1 x_2 + x_3 x_4 + x_5 x_6 + x_7 x_8 + x_9 x_{10})\ \\ \text{ subject } \text{ to }\\ 6 P l - \sigma x_9 x_{10}^2 \le 0\\ 6 P 2 l - \sigma x_7 x_8^2\le 0 \\ 6 P 3 l - \sigma x_5 x_6^2\le 0 \\ 6 P 4 l - \sigma x_3 x_4^2\le 0 \\ 6 P 5 l - \sigma x_1 x_2^2\le 0 \\ \frac{P l^3}{E} (244 x_3 x_4^3 x_5 x_6^3 x_7 x_8^3 x_9 x_{10}^3 +\\ +148 x_1 x_2^3 x_5 x_6^3 x_7 x_8^3 x_9 x_{10}^3 + 76 x_1 x_2^3 x_3 x_4^3 x_7 x_8^3 x_9 x_{10}^3 +\\ +28 x_1 x_2^3 x_3 x_4^3 x_5 x_6^3 x_9 x_{10}^3 + 4 x_1 x_2^3 x_3 x_4^3 x_5 x_6^3 x_7 x_8^3) \\ - \delta x_1 x_2^3 x_3 x_4^3 x_5 x_6^3 x_7 x_8^3 x_9 x_{10}^3\le 0 \\ x_2 - 20 x_1\le 0 \\ x_4 - 20 x_3\le 0 \\ x_6 - 20 x_5\le 0 \\ x_8 - 20 x_7\le 0 \\ x_{10} - 20 x_9\le 0, \end{array} \right. \end{aligned}$$

where $x_1 \in \{1,2,\ldots ,5\}$, $x_2,x_4 \in \{45,50,55,60\}$, $x_3,x_5 \in \{2.4, 2.6, 2.8, 3.1 \}$, $x_6 \in \{30,31,\ldots ,65\}$, $1 \le x_7 \le 5$, $30\le x_8, x_{10} \le 65$, $1\le x_9 \le 5$.

Besides, $P=50000$, $l=100$, $\delta = 2.7$, $\sigma = 14000$, $E = 2 \cdot 10^7$.

1.19 A.19 Problem MV1 (Yokota et al. 1998; Kumar et al. 2020): four-stage gear box problem

The problem minimizes the weight of a gear box where all variables are discrete.

Let $c_i = \sqrt{(y_{gi}-y_{pi})^2 + (x_{gi}-x_{pi})^2}$, $K_0 = 1.5$, $d_{\min } = 25$, $J_R = 0.2$, $\phi = 120$, $W=55.9$, $K_m=1.6$, $CR_{\min }=1.4$, $L_{\max }=127$, $C_p=464$, $\sigma _H=3290$, $\omega _{\max }=255$, $\omega _1 = 5000$, $\sigma _N=2090$ and $\omega _{\min }=245$,

$$\begin{aligned} \left\{ \begin{array}{ll} \min \left( \tfrac{\pi }{1000}\right) \sum _{i=1}^4 \tfrac{b_i c_i^2 (N_{pi}^2+N_{gi}^2)}{(N_{pi}+N_{gi})^2}\\ \text{ subject } \text{ to }\\ \left( \tfrac{366000}{\pi \omega _1} + \tfrac{2c_1N_{p1}}{N_{p1}+N_{g1}}\right) \left( \tfrac{(N_{p1}+N_{g1})^2}{4b_1 c_1^2 N_{p1}}\right) - \tfrac{\sigma _N J_R}{0.0167WK_0K_m} \le 0\\ \left( \tfrac{366000N_{g1}}{\pi \omega _1 N_{p1}} + \tfrac{2c_2N_{p2}}{N_{p2}+N_{g2}}\right) \left( \tfrac{(N_{p2}+N_{g2})^2}{4b_2 c_2^2 N_{p2}}\right) - \tfrac{\sigma _N J_R}{0.0167WK_0K_m} \le 0\\ \left( \tfrac{366000N_{g1}N_{g2}}{\pi \omega _1 N_{p1}N_{p2}} + \tfrac{2c_3N_{p3}}{N_{p3}+N_{g3}}\right) \left( \tfrac{(N_{p3}+N_{g3})^2}{4b_3 c_3^2 N_{p3}}\right) - \tfrac{\sigma _N J_R}{0.0167WK_0K_m} \le 0\\ \left( \tfrac{366000N_{g1}N_{g2}N_{g3}}{\pi \omega _1 N_{p1}N_{p2}N_{p3}} + \tfrac{2c_4N_{p4}}{N_{p4}+N_{g4}}\right) \left( \tfrac{(N_{p4}+N_{g4})^2}{4b_4 c_4^2 N_{p4}}\right) - \tfrac{\sigma _N J_R}{0.0167WK_0K_m} \le 0\\ \left( \tfrac{366000}{\pi \omega _1} + \tfrac{2c_1N_{p1}}{N_{p1}+N_{g1}}\right) \left( \tfrac{(N_{p1}+N_{g1})^3}{4b_1 c_1^2 N_{g1} N_{p1}^2}\right) - \left( \tfrac{\sigma _H}{C_p}\right) ^2 \left( \tfrac{\sin (\phi )\cos (\phi )}{0.0334WK_0K_m}\right) \le 0\\ \left( \tfrac{366000N_{g1}}{\pi \omega _1 N_{p1}} + \tfrac{2c_2N_{p2}}{N_{p2}+N_{g2}}\right) \left( \tfrac{(N_{p2}+N_{g2})^3}{4b_2 c_2^2 N_{g2} N_{p2}^2}\right) - \left( \tfrac{\sigma _H}{C_p}\right) ^2 \left( \tfrac{\sin (\phi )\cos (\phi )}{0.0334WK_0K_m}\right) \le 0\\ \left( \tfrac{366000N_{g1}N_{g2}}{\pi \omega _1 N_{p1}N_{p2}} + \tfrac{2c_3N_{p3}}{N_{p3}+N_{g3}}\right) \left( \tfrac{(N_{p3}+N_{g3})^3}{4b_3 c_3^2 N_{g3} N_{p3}^2}\right) - \left( \tfrac{\sigma _H}{C_p}\right) ^2 \left( \tfrac{\sin (\phi )\cos (\phi )}{0.0334WK_0K_m}\right) \le 0\\ \left( \tfrac{366000N_{g1}N_{g2}N_{g3}}{\pi \omega _1 N_{p1}N_{p2}N_{p3}} + \tfrac{2c_4N_{p4}}{N_{p4}+N_{g4}}\right) \left( \tfrac{(N_{p4}+N_{g4})^3}{4b_4 c_4^2 N_{g4} N_{p4}^2}\right) - \left( \tfrac{\sigma _H}{C_p}\right) ^2 \left( \tfrac{\sin (\phi )\cos (\phi )}{0.0334WK_0K_m}\right) \le 0\\ -N_{pi} \sqrt{\tfrac{sin^2(\phi )}{4} - \tfrac{1}{N_{pi}} + \left( \tfrac{1}{N_{pi}}\right) ^2} + N_{gi} \sqrt{\tfrac{sin^2(\phi )}{4} + \tfrac{1}{N_{gi}} + \left( \tfrac{1}{N_{gi}}\right) ^2} + \tfrac{\sin (\phi )(N_{pi}+N_{gi})}{2}\\ \hspace{1cm} + CR_{\min }\pi \cos (\phi ) \le 0, \forall i \in \{1,2,3,4\}\\ d_{\min } - \tfrac{2c_iN_{pi}}{N_{pi}+N_{gi}} \le 0, \forall i \in \{1,2,3,4\}\\ d_{\min } - \tfrac{2c_iN_{gi}}{N_{pi}+N_{gi}} \le 0, \forall i \in \{1,2,3,4\} x_{p1} + \tfrac{(N_{p1}+2)c_1}{N_{p1}+N_{g1}} - L_{\max } \le 0\\ - L_{\max } + \tfrac{(N_{pi}+2)c_i}{N_{pi}+N_{gi}} + x_{g(i-1)} \le 0, \forall i \in \{2,3,4\} - x_{p1} + \tfrac{(N_{p1}+2)c_1}{N_{p1}+N_{g1}} \le 0\\ \tfrac{(N_{pi}+2)c_i}{N_{pi}+N_{gi}} - x_{g(i-1)} \le 0, \forall i \in \{2,3,4\} y_{p1} + \tfrac{(N_{p1}+2)c_1}{N_{p1}+N_{g1}} - L_{\max } \le 0\\ - L_{\max } + \tfrac{(N_{pi}+2)c_i}{N_{pi}+N_{gi}} + y_{g(i-1)} \le 0, \forall i \in \{2,3,4\}\\ \tfrac{(N_{p1}+2)c_1}{N_{p1}+N_{g1}} - y_{p1} \le 0\\ \tfrac{(N_{pi}+2)c_i}{N_{pi}+N_{gi}} - y_{g(i-1)} \le 0, \forall i \in \{2,3,4\}\\ - L_{\max } + \tfrac{(N_{gi}+2)c_i}{N_{pi}+N_{gi}} + x_{gi} \le 0, \forall i \in \{1,2,3,4\}\\ - x_{gi} \tfrac{(N_{gi}+2)c_i}{N_{pi}+N_{gi}} \le 0, \forall i \in \{1,2,3,4\}\\ y_{gi} + \tfrac{(N_{gi}+2)c_i}{N_{pi}+N_{gi}} - L_{\max } \le 0, \forall i \in \{1,2,3,4\}\\ - y_{gi} + \tfrac{(N_{gi}+2)c_i}{N_{pi}+N_{gi}} \le 0, \forall i \in \{1,2,3,4\}\\ - (b_i - 8.255)(b_i - 5.715)(b_i - 12.70)(-N_{pi} + 0.945c_i -N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ (b_i - 8.255)(b_i - 3.175)(b_i - 12.70)(-N_{pi} + 0.646c_i -N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ (b_i - 5.715)(b_i - 3.175)(b_i - 12.70)(-N_{pi} + 0.504c_i -N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ (b_i - 5.715)(b_i - 3.175)(b_i - 8.255)(-N_{pi} -N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ - (b_i - 8.255)(b_i - 5.715)(b_i - 12.70)(N_{pi} - 1.812c_i + N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ (b_i - 8.255)(b_i - 3.175)(b_i - 12.70)(N_{pi} - 0.945c_i + N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ - (b_i - 5.715)(b_i - 3.175)(b_i - 12.70)(N_{pi} - 0.646c_i + N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ (b_i - 5.715)(b_i - 3.175)(b_i - 8.255)(N_{pi} - 0.504c_i + N_{gi}) \le 0, \forall i \in \{1,2,3,4\}\\ \omega _{\min } - \tfrac{\omega _1(N_{p1}N_{p2}N_{p3}N_{p4})}{(N_{g1}N_{g2}N_{g3}N_{g4})} \le 0\\ \tfrac{\omega _1(N_{p1}N_{p2}N_{p3}N_{p4})}{(N_{g1}N_{g2}N_{g3}N_{g4})} - \omega _{\max } \le 0. \end{array} \right. \end{aligned}$$