Animorphic ensemble optimization: a large-scale island model

Abstract

In this paper, a flexible large-scale ensemble-based optimization algorithm is presented for complex optimization problems. According to the no free lunch theorem, no single optimization algorithm demonstrates superior performance across all optimization problems. Therefore, with the animorphic ensemble optimization (AEO) algorithm presented here, a set of algorithms can be used as an ensemble which demonstrate stronger performance across a wider range of optimization problems than any standalone algorithm. AEO is a high-level ensemble designed to handle large ensembles using a well-defined stochastic migration process. The high-level nature of AEO allows for an arbitrary number of diverse standalone algorithms to interface with one another through an island model interface strategy, where various populations change size according to the performance of the algorithm associated with each population. In this study, AEO is demonstrated using ensembles of both evolutionary and swarm algorithms such as differential evolution, particle swarm, gray wolf optimization, moth-flame optimization, and more, and strong performance is observed. Quantitative diagnostics metrics to describe the migration of individuals across populations are also presented and observed with application to some test problems. In the end, AEO demonstrated strong consistent performance across more than 150 benchmark functions of 10–50 dimensions.

Appendices

Appendix A: Derivation of migration probabilities

In this section, the derivation for the expression of $M_{j,i}$ shown in Eq. (21) is given. To begin, the event where e members are exported from population i as decided in the debt settlement step of the migration phase is represented with $e_i$. As shown in Eq. (12), this event occurs according to the binomial distribution. Therefore, the probability of $e_i$ occurring can be described as.

$$\begin{aligned} \text {Pr}(e_i) = \left( \begin{array}{c}{N_i}\\ {e_i}\end{array}\right) h_i^{e_i}\left( 1 - h_i \right) ^{N_i - e_i}. \end{aligned}$$

(A.1)

Here, $h_i$ is defined in Eq. (13).

Next, $\vec {x} \in P_i \rightarrow$ can be defined as the event where a member of $P_i$ is exported. As mentioned previously in Sect. 3.2, the members within each population are treated identically only for the construction of the migration matrix. Furthermore, the probability of $\vec {x} \in P_i \rightarrow$ is dependent on the number of members being exported. Therefore, it may be useful to express

$$\begin{aligned} \text {Pr}(\vec {x} \in P_i \rightarrow | e_i) = \frac{e_i}{N_i}. \end{aligned}$$

(A.2)

From here, it may be fairly straightforward to see that

$$\begin{aligned} \text {Pr}(\vec {x} \in P_i \rightarrow ) = {\mathop {\sum \limits _{k=0}^{N_i}}} \frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) h_i^{k}\left( 1 - h_i \right) ^{N_i - k}. \end{aligned}$$

(A.3)

The final step in this derivation involves expressing the probability of $\vec {x} \in P_i \rightarrow P_j$, defined in Sect. 3.2 as the event where a member who begins in population i at the beginning of the migration phase ends up in population j at the end of the migration phase. To start this final statement, the case where $i \ne j$ can be expressed as:

$$\begin{aligned}&\text {Pr}(\vec {x} \in P_i \rightarrow P_j) = \frac{g_j^\alpha }{\sum g_k^\alpha }{\mathop {\sum \limits _{k=0}^{N_i}}} \frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) h_i^{k}\left( 1 - h_i \right) ^{N_i - k}\nonumber \\&\quad \text {where}\quad i \ne j. \end{aligned}$$

(A.4)

$\text {Pr}(\vec {x} \in P_i \rightarrow P_j)$ where $i = j$ is composed to two events. First, the event where an individual is not exported from its original population and second, the event where an individual is exported but returns to its original population. As such, $\text {Pr}(\vec {x} \in P_i \rightarrow P_j )$ where $i = j$ can be expressed as:

$$\begin{aligned}&\text {Pr}(\vec {x} \in P_i \rightarrow P_j ) = \left( 1 - {\mathop {\sum \limits _{k=0}^{N_i}}} \frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) h_i^{k}\left( 1 - h_i \right) ^{N_i - k}\right) \nonumber \\&\quad + \frac{g_j^\alpha }{\sum g_k^\alpha }{\mathop {\sum \limits _{k=0}^{N_i}}} \frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) h_i^{k}\left( 1 - h_i \right) ^{N_i - k}\; \text {where}\quad i \ne j. \end{aligned}$$

(A.5)

These two probabilities, where $i = j$ and where $i \ne j$, can be combined using the Kronecker delta function to aid in notation:

$$\begin{aligned}&\text {Pr}(\vec {x} \in P_i \rightarrow P_j ) = \delta _{ij} + \left( \frac{g_j^\alpha }{\sum g_k^\alpha } -\delta _{ij}\right) \nonumber \\&\quad {\mathop {\sum }\limits _{k=0}^{N_i}}\frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) \left( h_i\right) ^{k}\left( 1 - h_i \right) ^{N_i - k}. \end{aligned}$$

(A.6)

With the addition of the condition where $N= 0$ as discussed in Sect. 3.2, the final expression for $M_{i,j}$ is:

$$\begin{aligned} M_{j,i} = {\left\{ \begin{array}{ll} \delta _{ij} + \left( \frac{g_j^\alpha }{\sum g_k^\alpha } -\delta _{ij}\right) {\mathop {\sum }\limits _{k=0}^{N_i}}\frac{k}{N_i}\left( \begin{array}{c}{N_i}\\ {k}\end{array}\right) \left( h_i\right) ^{k}\left( 1 - h_i \right) ^{N_i - k} &{} N_i > 0 \\ \frac{1}{I} &{} N_i = 0 \end{array}\right. }. \end{aligned}$$

(A.7)

Appendix B: Benchmark functions

This appendix will describe the 52 benchmark functions which are used to evaluate AEO in this manuscript. The first set of functions to be discussed is the 22 classic functions. These functions were selected by the authors from various places in the literature. Table 5 gives information and references on these functions.

The CEC 2017 benchmark functions are organized and reported in [57]. The minima of these functions is given in Table 6. Evaluations of these functions are constrained to $[-100, 100]^d$ where d is the dimension of the input evaluation point.

Table 5 Characteristics and references for 22 benchmark functions used in the “classic” set

Table 6 Minima of benchmarks from CEC 2017 benchmarks

Appendix C: Optimization algorithm parameters

Many optimization algorithms have parameters which one can change to modify the behavior of the algorithm. In practice, selection of these parameters can be difficult due to the need for trial-and-error approaches to be used—often theoretical approaches to selecting these parameters may be limited in application. Nevertheless, algorithms can still perform well without tailoring these algorithm parameters to the specific problem. Furthermore, some algorithms such as GWO and WOA do not contain any of these algorithm parameters (excluding population size parameters). To maintain fairness between algorithms, the inherent performance of the algorithms in this study is evaluated with the same parameters across all benchmarks and these parameters are listed in Table 7. Explanations for the individual parameters associated with these algorithms are given in Sect. 2.1.

Also, the population sizes associated with each algorithm could be considered algorithm parameters. Population size has a similar meaning across all standalone algorithms discussed in this paper as the number of $\vec {x}$ which are maintained during the optimization process. Although the values and fitnesses of individuals within these populations may change, their size will remain constant. The population sizes used for each of the standalone algorithms are also listed in Table 7. For AEO as presented in this paper, the number of individuals for each of these populations listed in this table is used as a starting population sizes for these algorithms when included in an ensemble.

Table 7 Parameters and population sizes used for optimization algorithms

Appendix D: AEO behavior on Rastrigin function with large ensemble

See Figs. 6, 7, 8, and 9.

Fig. 6

Mobility metrics and population sizes for a large ensemble used to optimize the Rastrigin function in 7 dimensions. Here, the population associated with DE rebounds from 0 members

Fig. 7

Mobility metrics and population sizes for a large ensemble used to optimize the Rastrigin function in 7 dimensions. Here, all populations have the same $g_i$ leading to no migration being performed at two portions of the optimization routine according to the condition described in the penultimate paragraph of Sect. 3.1

Fig. 8

Mobility metrics and population sizes for a large ensemble used to optimize the Rastrigin function in 7 dimensions. Here, there are large population imbalances from cycle 30 to 55 which eventually level out

Fig. 9

Mobility metrics and population sizes for a large ensemble used to optimize the Rastrigin function in 7 dimensions. Here, erratic changes in population sizes can be observed all the way through the optimization process