1 Introduction

Differential Evolution (de) is one of the most widely used meta-heuristics to deal with continuous optimisation problems [17]. Due to its simplicity and efficiency, it has not only been applied to benchmark problems, but also to a wide range of real-world applications regarding electrical and power systems, robotics, and bio-informatics, among others [2]. Moreover, de and its variants have usually been one of the best performing approaches in different contests, such as the competition on Large Scale Global Optimisation (lsgo) organised in different editions of the Congress on Evolutionary Computation (cec) [10].

Regarding large scale problems, i.e., problems with a large dimensionality, typically more than 100 decision variables [6], a significant number of works have tried to modify different aspects of de for searching the vast decision space in a more efficient way [2]. For instance, in [12], a novel proposal that groups dependent decision variables in different sets was combined with de, being the latter responsible for optimising each set of variables separately. Another work proposed a linearly scalable exponential crossover operator that provided promising results considering a recent set of scalable benchmarks [19]. Recently, two novel schemes that improve the trial vector generation strategy of de were proposed, which showed to increase the performance of that algorithm when dealing with large scale problems [13]. Finally, the analysis of different strategies for initialising the population of de with the aim of improving its performance with large scale optimisation problems has gained a noticeable popularity in recent years [5, 18].

Some controversies have arisen concerning the benefits that a particular initialisation strategy, applied together with de, might provide when solving large scale problems [5]. The common belief is that some initialisation strategies can improve the performance of de for solving problems with a high dimensionality. For instance, the main conclusion given in [6] is that, in opposition to the application of basic random number generators as initialisation strategies, other more advanced methods should be considered in order to increase the performance of de when dealing with large scale problems, with the most suitable initialiser depending on the problem at hand. In a more recent work [5] however, authors claimed that the initialisation approach does not significantly affect the performance of de. In that paper, the behaviour of several advanced initialisation methods with different features was analysed when dealing with a set of large scale problems. Those initialisation strategies were combined with the best performing configuration found for one of the most widely used de variants. Although a few differences among initialisation techniques appeared in some cases when functions were analysed separately, all initialisation approaches performed in a statistically similar fashion taking into account the test suite as a whole.

In the current work, we try to shed some light on the above, by performing a novel study which consists of analysing the behaviour of a wide range of initialisation strategies in the overall, best, and worst cases. Those initialisation mechanisms are applied with the aforementioned best parameterisation of de to the same set of large scale problems considered in [5]. We show that initialisation strategies present a considerably larger number of statistically significant differences for the best and worst cases in comparison to the overall case, and therefore, a proper mechanism should be selected with the aim of increasing the probability of appearance of high-quality results and/or reducing the probability of appearance of low-quality ones, especially in cases where a high number of executions is not feasible.

The rest of this paper is organised as follows. de and the particular variant applied herein are described in Sect. 2. Section 3 is focused on introducing the initialisation strategies considered for our study. Afterwards, in Sect. 4, the different experiments conducted are exposed, together with their discussion. Finally, some conclusions and lines of future work are shared in Sect. 5.

2 Differential evolution

de is a stochastic direct search method especially suited for continuous global optimisation [17]. In de, the decision variables of a given problem are defined by a vector \(\mathbf {X} = [x_1, x_2, \ldots , x_i, \ldots , x_D]\), being D the number of decision variables or the dimensionality of the problem, and every \(x_{i = 1 \ldots D}\) a real number. As we previously mentioned, the term large scale problems is used to refer to those optimisation problems with a large dimensionality, typically \(D>100\). The quality of each vector \(\mathbf {X}\) is given by the objective function \(f(\mathbf {X})(f:\varOmega \subseteq \mathbb {R}^{D} \rightarrow \mathbb {R})\). The goal of the global optimisation, considering a minimisation problem, is thus to find a vector \(\mathbf {X*} \in \varOmega \) where \(f(\mathbf {X*}) \le f(\mathbf {X})\) holds for all \(\mathbf {X} \in \varOmega \). In the particular case of box-constrained continuous optimisation problems, the feasible region \(\varOmega \) is defined by particular values for the lower (\(a_i\)) and upper (\(b_i\)) bounds of each variable, i.e., \(\varOmega = \prod _{i=1}^{D} [a_i, b_i]\).

Taking into account the most widely used nomenclature for de [17], i.e., de /x/y/z, where x is the vector to be mutated, y defines the number of difference vectors used, and z indicates the crossover scheme, in this work we applied the approach de/rand/1/bin. We selected this variant due to its simplicity and popularity. The operation of this de variant is as follows. First of all, a population \(P = [\mathbf {X}_1, \mathbf {X}_2, \ldots , \mathbf {X}_j, \ldots , \mathbf {X}_{NP}]\) with NP individuals, also called vectors in the field of de, is initialised by using a particular strategy. Each individual comprises D decision variables. The value of the decision variable i belonging to the individual \(\mathbf {X}_j\) is denoted by \(x_{j,i}\). Then, successive iterations are evolved by executing the following steps. For each vector \(\mathbf {X}_j\) in the current population, called target vector, a new mutant vector \(\mathbf {V}_j\) is created using a mutant vector generation strategy. Several mutant vector generation strategies have been devised [2]. In our case, we applied the rand/1 scheme, which is probably the most popular one. The mutant vector \(\mathbf {V}_j\) for target vector \(\mathbf {X}_j\) is thus created as shown in Eq. 1, where \(r_1\), \(r_2\), and \(r_3\) are mutually exclusive integers chosen at random from the range [1, NP]. Furthermore, they are all different from the index j. The mutation scale factor F allows the exploration and exploitation abilities of de to be balanced.

$$\begin{aligned} \mathbf {V}_j = \mathbf {X}_{r_3} + F \times (\mathbf {X}_{r_1} - \mathbf {X}_{r_2}) \end{aligned}$$
(1)

After applying the mutant vector generation strategy, the mutant vector is combined with the target vector to generate the trial vector \(\mathbf {U}_j\) through a crossover operator. The combination of the mutant vector generation strategy and the crossover operator is usually referred to as the trial vector generation strategy. The most commonly applied operator for combining the target and mutant vectors—and the one considered in this paper—is the binomial crossover (bin). The crossover operation is controlled by means of the crossover rate CR. The binomial crossover generates a trial vector as shown in Eq. 2. A uniformly distributed random number in the range [0, 1] is given by \(rand_{j,i}\), and \(i_{rand} \in [1, 2, \dots , D]\) is an index selected in a random way that ensures that at least one variable is propagated from the mutant vector to the trial one. For the remaining cases, the probability of the variable being inherited from the mutant vector is CR. Otherwise, the variable of the target vector is taken into consideration.

$$\begin{aligned} u_{j,i} = \left\{ \begin{array}{ll} v_{j,i} &{}\quad if\ (rand_{j,i}\ \le \ CR\ or\ i\ =\ i_{rand}) \\ x_{j,i} &{}\quad otherwise \end{array} \right. \end{aligned}$$
(2)

The trial vector generation strategy, as described above, might generate vectors outside the feasible region \(\varOmega \). One of the most widely used schemes is based on randomly reinitialising the infeasible values in their corresponding feasible ranges, and it is the one applied herein. Finally, after generating NP trial vectors, each one is compared against its corresponding target vector. For each pair, the one that minimises the objective function is selected to survive. In case of a tie, in our implementation the trial vector survives.

3 Initialisation strategies for differential evolution

A wide range of initialisation strategies have been proposed in order to improve the results obtained by de [2, 5, 18]. In the current work, we compared the same set of initialisation strategies considered in [5], which are introduced herein.

Pseudo-Random Number Generators (prngs) and Chaotic Number Generators (cngs) are one of the most frequently used approaches for initialising a population of individuals [11, 15]. In the case of prngs, one of the most popular methods is Mersenne Twister (mt) [9], which is included as a typical prng on a large number of programming languages. Particularly, we used the variant that provides a period of \(2^{19937}\) and 623-dimensional equidistribution with 32-bit accuracy. Regarding cngs, we considered Tent Map (tm) [3]. This approach produces a chaotic sequence of numbers uniformly distributed in the range [0, 1], and has shown some benefits, like a higher iterative speed, with respect to other cngs, such as Logistic Map [3].

The aforementioned types of schemes take into account both randomness and uniformity to generate the initial population. There exist other kinds of schemes however, that only consider uniformity, and therefore, are usually deterministic. From among those strategies, we applied the methods Sobol Set (ss) [1] and Good Lattice Points (glp) [16], which are able to provide a set of points well distributed in the decision space.

Finally, Opposition-based Learning (obl) mechanisms as initialisation methods for de have gained a significant popularity in recent years [18]. Instead of considering randomness and/or uniformity, obl generates an initial population and calculates the opposite one with the aim of selecting the fittest individuals from both populations as the starting set. There are different variants of obl schemes [18]. In addition to the approaches considered in [5], i.e., obl and Quasi-Opposition-based Learning (qobl), we also applied Quasi-Reflection Opposition-based Learning (qrobl) herein, since a recent work [4] stated that the quasi-reflected opposition individual is more likely to be closer to the optimal solution than the opposition and quasi-opposition individuals.

4 Experimental evaluation

This section is devoted to describe the experiments conducted with the version of de introduced in Sect. 2 integrated with the different initialisation strategies depicted in Sect. 3.

Experimental method

The approach de/rand/1/bin, as well as the initialisation strategies considered, were implemented by using the Meta-heuristic-based Extensible Tool for Cooperative Optimisation (metco) [7]. Tests were run on Teide High Performance Computing facilities, which are composed of 1100 Fujitsu® computer servers, with a total of 17800 computing cores and 36 tb of memory. Since all experiments used stochastic algorithms, each execution was repeated \(numRep = 3 \times 10^3\) times, with the aim of comparing the different initialisation strategies with enough statistical confidence. With respect to the former, comparisons were carried out by applying the following statistical analysis [14]. First, a Shapiro-Wilk test was performed to check whether the values of the results followed a normal (Gaussian) distribution or not. If so, the Levene test checked for the homogeneity of the variances. If the samples had equal variance, an anova test was done. Otherwise, a Welch test was performed. For non-Gaussian distributions, the non-parametric Kruskal-Wallis test was used. For all tests, a significance level \(\alpha = 0.05\) was considered.

Problem set

Experiments were carried out using a set of scalable continuous optimisation problems proposed in cec’13 for its lsgo competition [8]. It is important to remark that this set of functions is the latest benchmark suite provided for large scale global optimisation in the field of the cec. Consequently, it was also considered for the lsgo competition organised during cec’15.Footnote 1 The set is composed of 15 functions (\(f_1\)–\(f_{15}\)) with different features: fully-separable functions (category 1: \(f_1\)–\(f_3\)), partially additively separable functions (category 2: \(f_4\)–\(f_{11}\)), overlapping functions (category 3: \(f_{12}\)–\(f_{14}\)), and a non-separable function (category 4: \(f_{15}\)). Following the indications given for the different editions of the lsgo competition, in the current work, the number of decision variables D was fixed to 1000 for all the aforementioned functions, with the exception of problems \(f_{13}\) and \(f_{14}\), where D was fixed to 905 decision variables due to overlapping subcomponents.

Parameters

The experiments conducted applied a common parameterisation for different configurations of the scheme de/rand/1/bin, which can be observed in Table 1. The only difference among configurations resides on the initialisation strategy used. In previous work [5], a configuration of the scheme de/rand/1/bin using those parameter values, from among a candidate pool with more than 80 different configurations of that approach, was able to provide the best overall results for problems \(f_1\)–\(f_{15}\) with 1000 decision variables. That is the reason why we have selected those values for the parameters NP, F, and CR. Finally, rules of the lsgo competition indicate that the stopping criterion has to be fixed to a maximum number of \(3 \times 10^6\) function evaluations. In order to perform the analyses, de was applied with each considered initialisation strategy to each of the 15 benchmark problems, thus giving a total number of \(3.15 \times 10^5\) runs. Following the recommendations given in [5, 10], Eq. 3 was applied to assign a seed s(i) to the i-th run of every de configuration, regardless of the initialisation method.

$$\begin{aligned} s(i) = i \ \ \ \forall i \in \{1, 2, \ldots , numRep\} \end{aligned}$$
(3)
Table 1 Different parameterisations of the scheme de/rand/1/bin
Full size table

Table 2 shows rankings of the considered initialisation strategies when the best 300 and the worst 300 executions, i.e., those with the lowest and highest values of the objective function at the end of the runs, respectively, are taken into account. Results regarding all the executions are also shown. In order to calculate rankings the following steps were performed. First, the number of approaches that a particular strategy statistically outperformed (\(\uparrow \)), as well as the number of times that it was statistically outperformed (\(\downarrow \)) by the remaining schemes, considering all problems, were calculated by applying the statistical procedure explained at the beginning of the current section. Approach A statistically outperforms scheme B if there exist statistically significant differences between them, i.e., if the p-value is lower than \(\alpha = 0.05\), and if at the same time, A provides a lower mean and median of the objective value than B, since we are dealing with minimisation problems. Afterwards, the score assigned to a strategy is given by the difference between the number of schemes it was able to beat and the number of schemes that were able to beat it. Finally, a ranking is established by sorting strategies in descending order taking into account the scores assigned.

If we consider the whole set of executions, the best performing overall approach was obl, although followed by mt with a similar score. In this case, it can be observed that scores assigned to both aforementioned approaches were lower than those assigned to the first-ranked scheme in the best and worst cases. This means that, if all executions are taken into account, the number of differences among initialisation strategies (122 cases out of 630) significantly decreases in comparison to the number of differences that appears regarding the best (234 cases) and the worst results (256 cases). Furthermore, since two methods obtained similar scores, no initialisation strategy was able to provide a clear advantage with respect to the remaining ones when considering the set of problems as a whole. As a result, it might seem that the initialisation technique does not affect the overall performance of de when solving large scale problems. The above agrees with the conclusions given in [5].

Table 2 Ranking of initialisation strategies considering the best 300 and the worst 300 executions for problems \(f_1\)–\(f_{15}\)
Full size table

Nevertheless, it can be observed that obl was the best performing initialisation strategy in cases when the best and the worst 300 executions were analysed, since it obtained significantly better scores than the remaining approaches. The method obl was statistically better in a larger number of cases than the remaining strategies, while it was statistically worse in a lower number of cases when compared to the remaining schemes. By the application of obl as an initialisation technique, we therefore might increase/reduce the probability of appearance of high-quality/low-quality executions when using de for solving large scale continuous optimisation problems. A more in-depth analysis of this method however, should be carried out for each problem, with the aim of providing more evidence of its advantages and drawbacks.

Table 3 Statistical comparison between obl and the remaining strategies considering problems \(f_1\)–\(f_{15}\) and all executions
Full size table

4.1 Analysis of the scheme obl considering all the results

This section focuses on comparing the approach obl with respect to the remaining strategies when considering all the executions. Table 3 shows, for each problem, the p-values obtained from the statistical comparison between the scheme obl and the remaining approaches. It also shows cases for which obl was able to statistically outperform other strategy (\(\uparrow \)), cases where other strategy outperformed obl (\(\downarrow \)), and cases where statistically significant differences between obl and the corresponding method did not arise (\(\leftrightarrow \)). Finally, in cases where statistically significant differences between obl and the corresponding scheme appeared, but one approach obtained the lowest mean, while the other one provided the lowest median, an ‘*’ is shown. It can be observed that in 10 out of 15 problems, no statistically significant differences appeared between obl and the remaining initialisation strategies. This confirms our previous statement concerning the lack of differences between initialisation methods when the overall results are considered. Generally speaking, there was no strategy that clearly provided better results than the remaining ones regardless of the addressed problem. Despite that, some differences arose in some cases. obl was better than several approaches when solving functions \(f_2\), \(f_{10}\), \(f_{12}\), and \(f_{15}\), with each one belonging to a different category. For instance, in the case of the non-separable function \(f_{15}\), obl showed a clear superiority together with mt. Only in the case of function \(f_2\) obl was statistically worse than another scheme (tm).

4.2 Analysis of the scheme obl considering the best results

This section is devoted to compare the initialisation scheme obl in regard to the remaining approaches when the best 300 executions are considered. Results of this analysis are shown in Table 4.

It is important to remark that, taking into account the best 300 executions, differences between obl and the remaining methods appeared in 11 out of 15 problems, being this a significant increase concerning the previous analysis of the overall results. obl did not present statistically significant differences with any other approach for functions \(f_4\), \(f_5\), \(f_9\), and \(f_{11}\). At the same time, in 8 out of 15 problems, obl was able to outperform other strategies, and in 4 out of those 8 functions, it was not worse than any other initialisation strategy. Considering function \(f_{14}\), for example, obl was statistically better, together with mt and ss, than the remaining schemes. This means that, if users would like to increase the probability of appearance of high-quality results when solving problem \(f_{14}\), they should initialise the population of de with one of those three strategies. Moreover, depending on the problem being solved, the most suitable initialisation strategy changes. For instance, taking into account function \(f_8\), the best performing scheme was obl, together with mt and glp, while in the case of \(f_{7}\), qobl provided the best performance.

Table 4 Statistical comparison between obl and the remaining strategies considering problems \(f_1\)–\(f_{15}\) and the best 300 executions
Full size table

4.3 Analysis of the scheme obl considering the worst results

In this section, we carry out a similar analysis than the one exposed in the previous section, but in this case, we compare the strategy obl with respect to the remaining approaches when considering the worst 300 executions. Results of this study are shown in Table 5.

Table 5 Statistical comparison between obl and the remaining strategies considering problems \(f_1\)–\(f_{15}\) and the worst 300 executions
Full size table

In the worst case, differences between obl and the remaining methods appeared in 14 out of 15 problems. As in the best case, this is a significant increase of differences in comparison to the study considering all executions. Only in the case of function \(f_8\), obl did not present statistically significant differences with any other approach. In 11 out of 15 problems, obl was able to outperform other strategies. In fact, in 10 out of those 11 functions, it was not worse than any other initialisation strategy. For instance, considering function \(f_2\), obl was statistically better, together with mt and tm, than the remaining schemes. This means that, in the worst case, de would attain better results for problem \(f_2\) by applying one of those three strategies. Additionally, depending on the problem being solved, as in the best case, the most suitable initialisation strategy changes. Taking into account function \(f_9\), for example, the best performing scheme was obl, together with tm, glp and qrobl, while in the case of \(f_{14}\), ss provided the best performance.

Finally, it is worth mentioning that, if we consider the best and worst cases simultaneously, there exist two problems (\(f_3\) and \(f_{15}\)) for which obl, and other schemes, were the best performing initialisation approaches. The above means that those initialisation strategies allow the probability of appearance of high-quality results to be increased, and at the same time, are able to reduce the probability of appearance of low-quality results when solving those two particular functions.

5 Conclusions and future work

Some controversies have arisen in recent years regarding the benefits of using a given initialisation strategy for solving large scale problems with de. Some works have stated that certain initialisation mechanisms are able to increase the performance of de. Other authors however, have claimed that the initialisation strategy does not significantly change the way de performs.

Analysing the overall case, we showed that differences among the initialisation strategies considered were not statistically significant for a considerable number of problems. Bearing the above in mind, it might seem that the initialisation technique does not generally affect the performance of de when solving large scale problems. However, when studying the best and worst cases, which had not been previously analysed, significant differences appeared among initialisation mechanisms, with obl being the best performing approach for a wide range of problems. As a result, with the aim of increasing the probability of appearance of high-quality results and/or decreasing the probability of appearance of low-quality solutions when de is used to solve large scale problems, a suitable initialisation strategy, which depends on the problem at hand, should be selected. For those cases where we do not have enough a priori information about the problem being solved, for instance, when dealing with black-box optimisation problems, obl seems to be a promising scheme. Finally, we should note that the above is even more important in those scenarios where only a few executions can be performed, for example, when solving large scale real-world problems with time-consuming evaluation functions.

Although qobl and qrobl are extensions of obl, the experimental evaluation carried out in this work showed that they were not able to provide better results than the latter. It is likely that this is because different variants of de, with different parameter values, components, and stopping criteria, are studied depending on the considered work. Another possibility might be the dimensionality used for defining the problems. Due to the above reasons, it would be interesting to study whether more sophisticated initialisation strategies, such as qrobl and qobl, among others, are able to provide some benefits with respect to the traditional obl scheme. Another line of future work would be to analyse the behaviour of different mechanisms based on obl, as well as other initialisation schemes, in the worst and best cases, by combining them with different variants of de applied to several sets of problems. This might allow the conclusions extracted in this work to be generalised.

Finally, we should note that another factor that could affect the quality of the algorithm initialisation is the population size. As it was already mentioned, a significant number of configurations of one of the most widely used de variants were analysed in a previous work considering the overall case. Those configurations were obtained by combining different values for the parameters of de, including the population size. The best performing configuration was applied with one of the biggest population sizes considered. In opposition to the resolution of problems with lower dimensionalities, where smaller populations perform better, the analyses carried out in the aforementioned work concluded that an increase of the population size to some threshold values is more suitable when dealing with large scale optimisation. Nevertheless, those studies also concluded that differences among the initialisation strategies considered were not statistically significant when using larger population sizes. Bearing the above in mind, and considering that the said best performing configuration was also applied in the current work, the effects that the population size, together with the initialisation strategy, may have on the quality of the algorithm initialisation, were not analysed herein. Since the overall case has already been analysed however, it would be very interesting to carry out a study about the performance of de through the combination of different initialisation strategies and different population sizes taking into account the best and worst cases.