A two-stage surrogate-assisted meta-heuristic algorithm for high-dimensional expensive problems

Abstract

This study proposes a two-stage surrogate-assisted meta-heuristic algorithm named SDAMA-SPS to solve computationally expensive problems with high dimensions. In this algorithm, a surrogate-assisted monkey algorithm with dynamic adaptation (SDAMA) is presented to globally search for the best solution of the first stage, and a surrogate-based perturbation search (SPS) is designed to perform a more intensive local search for the final optimal solution. In the first stage, a global radial basis function (RBF) surrogate model is constructed with all historical solutions and is used to evaluate the positions of monkeys in the climb process and watch-jump process. Such global RBF model is updated with the positions of monkeys and their real objective function values after the second round of the climb process for each iteration. In the second stage, a local RBF surrogate model is built with a set of current best solutions, which can help to select the most promising sample so as to further locally improve the solution searched in the first stage. Experimental studies are conducted on eight benchmark optimization problems with the number of dimensions varying from 30 to 100, and numerical results show that the proposed algorithm achieves better performance than five other state-of-the-art surrogate-assisted algorithms with a limited budget of function evaluations.

Appendix

1.1 Appendix A: Weighted scoring criterion

1.1.1 Step 1 prediction score

• (a) Use the RBF model $s_{n} \left( {\varvec{x}} \right)$ (n is the number of evaluated points) to estimate all candidate points (denoted by CPs). Additionally, compute $s_{n}^{{{\text{max}}}} = {\text{max}}\left\{ {s_{n} \left( {\varvec{x}} \right):{\varvec{x}} \in {\varvec{CPs}}} \right\}$ and $s_{n}^{{{\text{min}}}} = {\text{min}}\left\{ {s_{n} \left( {\varvec{x}} \right):{\varvec{x}} \in {\varvec{CPs}}} \right\}$.

• (b) For each candidate point ${\varvec{x}} \in {\varvec{CPs}}$, compute its prediction score $s_{n}^{pred} \left( {\varvec{x}} \right) = (s_{n} \left( {\varvec{x}} \right) - s_{n}^{{{\text{min}}}} )/\left( {s_{n}^{{{\text{max}}}} - s_{n}^{{{\text{min}}}} } \right)$ if $s_{n}^{min} \ne s_{n}^{max}$ and $s_{n}^{pred} \left( {\varvec{x}} \right) = 1$, otherwise.

1.1.2 Step 2 Distance score

• (a) Determine the minimum distance of each candidate point from previously evaluated points (denoted by EPs). For each ${\varvec{x}} \in {\varvec{CPs}}$, compute $\Delta_{n} \left( {\varvec{x}} \right) = {\text{min}}D\left( {{\varvec{x}}, {\varvec{x}}_{i} } \right)$, where ${\varvec{x}}_{i} \in {\varvec{EPs}}$. Additionally, compute $\Delta_{n}^{{{\text{max}}}} = {\text{max}}\left\{ {\Delta_{n} \left( {\varvec{x}} \right):{\varvec{x}} \in {\varvec{CPs}}} \right\}$ and $\Delta_{n}^{{{\text{min}}}} = {\text{min}}\left\{ {\Delta_{n} \left( {\varvec{x}} \right):{\varvec{x}} \in {\varvec{CPs}}} \right\}$.

• (b) For each candidate point ${\varvec{x}} \in {\varvec{CPs}}$, compute its distance score $s_{n}^{{{\text{dist}}}} \left( {\varvec{x}} \right) = (\Delta_{n}^{{{\text{max}}}} - \Delta_{n} \left( {\varvec{x}} \right))/(\Delta_{n}^{{{\text{max}}}} - \Delta_{n}^{{{\text{min}}}} )$ if $\Delta_{n}^{{{\text{min}}}} \ne \Delta_{n}^{{{\text{max}}}}$ and $s_{n}^{{{\text{dist}}}} \left( {\varvec{x}} \right) = 1$, otherwise.

1.1.3 Step 3 Determine the cyclical weights

$$ \omega = \left\{ {\rho_{1} ,\rho_{2} \ldots ,\rho_{h} } \right\},0 \le \rho_{1} \le \rho_{2} \le \cdot\cdot\cdot \le \rho_{h} \le 1 $$

$Set \omega_{n}^{pred} = \left\{ {\begin{array}{*{20}c} {\rho_{{{\text{mod}}\left( {n,h} \right)}} , if {\text{mod}}\left( {n,h} \right) \ne 0} \\ {\rho_{h} , otherwise} \\ \end{array} } \right.$ and $\omega_{n}^{{{\text{dist}}}} = 1 - \omega_{n}^{{{\text{pred}}}}$.

1.1.4 Step 4 Compute the weighted score

For each candidate point ${\varvec{x}} \in {\varvec{CPs}}$, compute the weighted score $W_{n} \left( {\varvec{x}} \right) = \omega_{n}^{{{\text{pred}}}} s_{n}^{{{\text{pred}}}} \left( {\varvec{x}} \right) + \omega_{n}^{{{\text{dist}}}} s_{n}^{{{\text{dist}}}} \left( {\varvec{x}} \right)$.

1.1.5 Step 5 select the next evaluation point

Let ${\varvec{x}}_{n + 1} \leftarrow arg\min \left( {W_{n} \left( {\varvec{x}} \right)} \right),{\varvec{x}} \in {\varvec{CPs}}$.

The reasons for designing such a weighted scoring criterion are briefly given as follows: Initially, we prefer the candidate points that are far away from ${{\varvec{x}}}_{best}$ to focus on extensive exploration (i.e., global search). Moreover, it is not sufficiently accurate for the initial RBF built with limited evaluation samples to predict the candidate points. Therefore, we set ${\omega }_{n}^{D}=1$ and ${\omega }_{n}^{P}=0$. Later, the RBF fitted by more evaluation samples will become more accurate, and in this condition, we can pay a higher weight for the prediction score to enhance the local search. Thus, ${\omega }_{n}^{D}$ and ${\omega }_{n}^{P}$ are reduced and increased, respectively, to balance the global and local search. In addition, we use the cycling method of adjusting both weights to avoid the danger of being trapped in local optima. That is, when ${\omega }_{n}^{D}$ reaches 0, we reset ${\omega }_{n}^{D}=1$. For more specific explanations and analyses, please refer to Regis and Shoemaker (2007, 2013).