Global optimization method using ensemble of metamodels based on fuzzy clustering for design space reduction

Abstract

For most engineering design optimization problems, it is difficult or even impossible to find the global optimum due to the unaffordable computational cost. To overcome this difficulty, a global optimization method that integrates the ensemble of metamodels and fuzzy clustering is proposed to deal with the optimization problems involving the computation-intensive, black-box computer analysis and simulation. The ensemble of metamodels combining three representative metamodeling techniques with optimized weight factors is used to decrease the computational expense during the optimization procedure and the fuzzy clustering technique is applied to obtain the reduced design space. In this way, the efficiency and capability of capturing the global optimum will be improved in the reduced design space. To demonstrate the superior performance of the proposed global optimization method over existing methods, it is examined using various benchmark optimization problems and applied to solve an engineering design optimization problem. The results show that the proposed global optimization method is robust and efficient in capturing the global optimum.

Appendix: Description of selected metamodeling techniques

1.1 Response surface method, RSM

Response surface method is one of the most popular established metamodeling techniques. The standard RSM first employs an experimental strategy to generate design points in the design space, then applies either a first-order model or a second-order model to approximate the unknown system. The most commonly used RSM model is the second-order model in the form of a second-degree algebraic polynomial function as:

$$\tilde{f}\left( x \right) = \beta_{0} + \sum\limits_{i = 1}^{n} {\beta_{i} \varvec{x}_{i} } + \sum\limits_{i = 1}^{n} {\beta_{ii} \varvec{x}_{i}^{2} } + \sum\limits_{i < j} {\sum\limits_{j = 1}^{n} {\beta_{ij} \varvec{x}_{i} \varvec{x}_{j} } }$$

(24)

where n is the number of variables in the input vector x; $\tilde{f}(\varvec{x})$ is the response surface approximation of the actual function. $\beta_{0} , \;\beta_{i} ,\;\beta_{ii} ,\;\beta_{ij}$ represent unknown regression coefficients which can be calculated by the least squares technique.

1.2 Radial basis function, RBF

The approximate method RBF is originally developed by Hardy in 1971 to fit irregular topographic contours of geographical data. It has been known tested and verified for several decades and many positive properties have been identified.

For a data set consisting of the values of design variables and response values at N sampling points, the true function $f(\varvec{x})$ can be approximated as

$$\tilde{f}\left( \varvec{x} \right) = \sum\limits_{i = 1}^{N} {\lambda_{i} \phi \left( {\left\| {\varvec{x - x}_{i} } \right\|} \right)}$$

(25)

where x is a vector of the design variables, x _i is a vector of the design variables at the ith sampling point, $\varvec{x} - \varvec{x}_{i} = \sqrt {\left( {\varvec{x} - \varvec{x}_{i} } \right)^{T} \left( {\varvec{x} - \varvec{x}_{i} } \right)}$ is the Euclidean norm, representing the radial distance r, from design point x to the sampling point x _i , $\phi$ is a basis function and λ _i is the coefficient for the ith basis function. The approximation function $\tilde{f}(\varvec{x})$ is actually a linear combination of some basis functions with weight coefficients λ _i . The most commonly used basis functions are listed in Table 5. In this study, the basis function multiquadrics are chosen because of its prediction accuracy and its commonly linear and possibly exponential rate of convergence with increased sampling points. Besides, the choice of c ₁ = 1 is found to be suitable for most function approximations which is used in this paper.

Table 5 Commonly used basis functions

1.3 Kriging

Kriging is named after the original work of South African engineer D.G. Krige. It estimates the value of a function as a combination of a known function $\tilde{f}_{i} (\varvec{x})$ (e.g., a linear model such as a polynomial trend) and departures (representing low and high frequency variation components, respectively,) of the form:

$$\tilde{f}\left( \varvec{x} \right) = \sum\limits_{i = 1}^{m} {\beta_{i} f_{i} \left( \varvec{x} \right)} + Z\left( \varvec{x} \right)$$

(26)

where $\beta_{i}$ is an unknown constant estimated, Z(x) is assumed to be a realization of a stochastic process with zero mean and a nonzero covariance. The i, jth element of covariance matrix of Z(x) is given by

$${\text{Cov}}\left[ {Z\left( {\varvec{x}^{i} } \right),Z\left( {\varvec{x}^{j} } \right)} \right] = \sigma_{z}^{2} R_{ij}$$

(27)

where $\sigma_{z}^{2}$ is the process variance, and R _ij is the correlation function between the ith and jth data points; The Gaussian function is used as the correlation function in this study, defined by

$$R\left( {\varvec{x}^{i} ,\varvec{x}^{j} } \right) = R_{ij} = \exp \left\{ { - \sum\limits_{k = 1}^{n} {\theta_{k} \left( {x_{k}^{i} - x_{k}^{j} } \right)}^{2} } \right\}$$

(28)

where $\theta_{k}$ is distinct for each dimension, and these unknown parameters are generally determined by solving a nonlinear optimization problem.