An environment-driven hybrid evolutionary algorithm for dynamic multi-objective optimization problems

Abstract

In dynamic multi-objective optimization problems, the environmental parameters may change over time, which makes the Pareto fronts shifting. To address the issue, a common idea is to track the moving Pareto front once an environmental change occurs. However, it might be hard to obtain the Pareto optimal solutions if the environment changes rapidly. Moreover, it may be costly to implement a new solution. By contrast, robust Pareto optimization over time provides a novel framework to find the robust solutions whose performance is acceptable for more than one environment, which not only saves the computational costs for tracking solutions, but also minimizes the cost for switching solutions. However, neither of the above two approaches can balance between the quality of the obtained non-dominated solutions and the computation cost. To address this issue, environment-driven hybrid dynamic multi-objective evolutionary optimization method is proposed, aiming to fully use strengths of TMO and RPOOT under various characteristics of environmental changes. Two indexes, i.e., the frequency and intensity of environmental changes, are first defined. Then, a criterion is presented based on the characteristics of dynamic environments and the switching cost of solutions, to select an appropriate optimization method in a given environment. The experimental results on a set of dynamic benchmark functions indicate that the proposed hybrid dynamic multi-objective evolutionary optimization method can choose the most rational method that meets the requirements of decision makers, and balance the convergence and robustness of the obtained non-dominated solutions.

Introduction

In many practical applications, such as scheduling [1], controller optimization [2, 3] and path planning [4, 5], we focus on more than one objective that contradicts each other. More especially, the key parameters, the number of objectives and constrains may change over time [6,7,8,9], forming dynamic multi-objective optimization problems (DMOPs) [10]. For example, in a hydro-thermal power generation systems, the total fuel cost of thermal generation and emission properties are minimized, while satisfying all constraints in the hydraulic and power system networks, so as to allocate power reasonably to the hydroelectric and thermal generating units. It is in essence a dynamic optimization problem due to the time-varying power demand [10]. Apparently, tracking the optimal solution in time as any dynamic factor occurs is a challenging task for DMOPs.

To address DMOPs, many meta-heuristic methods [11,12,13,14,15,16] have been introduced that can be categorized into two classes. One is to re-trigger the multi-objective optimization process as a new environment appears, with the purpose of finding the Pareto-optimal solutions more close to the new true Pareto front as soon as possible. We call this approach tracking moving Pareto-optimum (TMO). The key issue on TMO is to keep the better diversity of population as a new environment appear. Rich studies have been done on change response techniques, including environmental detection [1, 6, 8], immigrant-learning [11], prediction and memory strategies [14, 15, 17,18,19,20]. All these prove that TMO is a powerful method to solve DMOPs with slowly-changing environments and less switching cost of solutions. However, in many real world optimization problems, tracking the new Pareto-optima every time the environment changes is impractical due to the limited computation resources for optimization or the expensive switching cost from the previous optima [21]. To save the computation and reduce switching times, Yu et al. [22] firstly proposed a novel framework to find robust optimum over time (ROOT) for dynamic scalar optimization problems, in which a robust solution implemented in the current situation also has the acceptable convergence and can be kept in use after the environment changes. Jin et al. [23] presented a quantitative description of robustness and a general framework for finding ROOT. Following that, Chen et al. [24] presented a practical definition of finding robust Pareto-optima over time (RPOOT) for DMOPs. A Pareto-optimal solution that was found in the current environment and has the acceptable convergence in the subsequent environments are termed as a robust candidate [25]. Apparently, reusing the robust solution in the successive environments saves the computation and switching cost. But the robustness of a candidate will become worse as the environment changes intensely [26]. Once no historical Pareto-optimal solution has the satisfactory performance on at least one objective in the sequence environments, TMO will be an effective alternative to RPOOT. Consequently, TMO and RPOOT are both successful problem-solvers for DMOPs, however, fit for various environmental changes.

In benchmark test suites for dynamic optimization, the environments change over time with different way, such as linear change [11, 27, 28], chaotic change [29], and stochastic change [30]. Moreover, the environmental change occurs with various frequencies and intensities. For example, in the dynamic scheduling problems of airport fuel filling vehicles [31] three objectives were taken into account, including minimizing the length of the whole routes and the number of running vehicles, as well as minimizing the task quantity difference between vehicles. Because the take-off and landing times of each aircraft are always changing in terms of weather, congestion situation and other dynamic factors that may occur irregularly, the refueling vehicles routing problem is essentially a DMOP with complex environmental changes, the corresponding frequency and intensity of environmental changes are not fixed. During finding the optimal scheduling scheme for all refueling vehicles under the uncertain delay time of the aircrafts, traditional TMO may not meet the needs of a real-time task due to the frequent adjustment of vehicle routing, while RPOOT can only provide a suboptimal scheme with robustness over time. Based on this, an environment-driven hybrid dynamic multi-objective evolutionary optimization algorithm is presented in this paper, with the purpose of effectively balancing the quality of Pareto-optima and switching cost. In the proposed method, a prediction model is built to estimate the frequency and intensity of environmental changes based on their historical information. Based on the characteristics of dynamic environments and the switching cost of solutions, a selection criterion is designed to adaptively trigger TMO or RPOOT.

The rest of the paper is organized as follows. The main ideas of TMO and RPOOT are illustrated in “The main ideas of TMO and RPOOT” section. “Indexes measuring environmental changes” section defines various parameters that represent the features of environmental changes. Based on this, a generic framework of hybrid dynamic multi-objective evolutionary optimization method is developed and the condition for selecting one of the two methods is given in “Proposed algorithm” section. In “Experimental study” section, the experimental results are analyzed under different environmental changes. Finally, a conclusion of the paper and future work are given in “Conclusion” section.

The main ideas of TMO and RPOOT

Without loss of generality, a DMOP with time-varying parameters can be described as follows.

$$\begin{aligned} \begin{array}{l} \min \pmb F(\pmb x,\pmb {\alpha }^t)=\left( f_1 (\pmb x,\pmb {\alpha }^t),\ldots ,f_m (\pmb x,\pmb {\alpha }^t)\right) \\ \text{ s.t. }\;\,\pmb x \in \Omega . \\ \end{array} \end{aligned}$$

(1)

where ${\pmb x}=(x_1,x_2,\ldots ,x_n)\in {\pmb {\Omega }}\subset {\pmb R}^{n}$ represents the decision vector and ${\pmb {\Omega }}$ is the n-dimensional decision space. ${\pmb F}=(f_1,f_2,\ldots ,f_m)\in {\pmb {\Lambda }}\subset {\pmb R}^{m}$ is the objective function vector and ${\pmb {\Lambda }}$ is the m-dimensional objective space. $\pmb {\alpha }^t$ denotes the environmental parameter changing over time. In practice, the environmental changes generally occur at some discontinuous time points, thus, the time-varying parameters can be discretized to $\pmb {\alpha }^k,k=1,2,\ldots ,N$. Assume that the dynamic parameters vary with frequency $f^k=1/\tau ^k$, and $\pmb {\alpha }^k$ remains constant when $t\in [\sum _{j=0}^{k-1}{\tau ^j},\sum _{j=0}^{k}{\tau ^j}]$. $\tau ^j$ is the duration of the jth environment and $\tau ^0=0$. Following that, a DMOP is transformed to N static MOPs, denoted as $\left\langle \min {\pmb F}({\pmb x},\pmb {\alpha }^1),\min {\pmb F}({\pmb x},\pmb {\alpha }^2),\ldots ,\min {\pmb F}({\pmb x},\pmb {\alpha }^N) \right\rangle$.

To solve the above-mentioned DMOPs, most research has focused on tracking the new Pareto front with a fast convergence speed once an environmental change occurs. More specifically, it is essential for a TMO-based dynamic evolutionary multi-objective algorithm to detect the environmental changes in time, generate an initial population using the knowledge acquired from the previous environments, and locate the new Pareto front as soon as possible. In particular, evolutionary algorithms and other meta-heuristics, such as multi-objective genetic algorithms, particle swarm optimization and differential evolution [11, 27, 32,33,34,35,36,37,38], have been developed to follow the new Pareto-optimal solutions quickly. A generic framework of TMO is shown in Algorithm 1.

Different from TMO, the goal of RPOOT-based dynamic multi-objective optimization algorithms is to find the robust Pareto-optimal solutions, expressed by RPS $=\left\langle \mathrm{RPS}(1)\right.$,$\left. \mathrm{RPS}(2),\ldots ,\mathrm{RPS}(L)\right\rangle (1\le L\le N)$, which have the acceptable performance in more than one environment. To this end, two indexes have been defined to evaluate the robustness of an individual [24]. One is robustness for optimality, which measures the degree of ${\pmb x}_i^j \in \mathrm{{RPS}}(i)$ approximate to the true Pareto fronts in T consecutive environments, denoted as ${\pmb F}^\mathrm{ave}({\pmb x}_i^j,\pmb {\alpha }^k)$. T is called the time window and preset by a decision maker according to his/her expectation for the adaptability of an individual to environmental changes.

$$\begin{aligned} {\pmb F}^\mathrm{ave}({\pmb x}_i^j,\pmb {\alpha }^k)&= \left( f_1^\mathrm{ave}({\pmb x}_i^j,\pmb {\alpha }^k),\ldots ,f_m^\mathrm{ave}({\pmb x}_i^j,\pmb {\alpha }^k)\right) \end{aligned}$$

(2)

$$\begin{aligned} f_q^\mathrm{ave}({\pmb x}_i^j,\pmb {\alpha }^k)&=\frac{1}{T}\left( f_q({\pmb x}_i^j,\pmb {\alpha }^k)+\sum _{l=1}^{T-1}{\hat{f_q}({\pmb x}_i^j,\pmb {\alpha }^{k+l})}\right) \end{aligned}$$

(3)

The other is the temporal robustness, denoted as $L_i$, which reflects the survival time of RPS(i) in the subsequent environments. Let $\eta$ be a preset threshold that represents the maximum acceptable increment of the fitness value. Denote ${\hat{{\pmb F}}}({\pmb x}_i^j,\pmb {\alpha }^{k+l({\pmb x}_i^j)}))$ as the estimated fitness of ${\pmb x}_i^j$ in the $k+l$ environments, one has

$$\begin{aligned}&L_i =\mathop {\min }\limits _{{\pmb x}_i^j \in \mathrm{{RPS}}(i)} \bar{l}({\pmb x}_i^j ) \end{aligned}$$

(4)

$$\begin{aligned}&\bar{l}({\pmb x}_i^j)\!=\!\max \! \left\{ \!{l({\pmb x}_i^j)}|\Delta (l({\pmb x}_i^j))\!\le \!\eta ,\!l({\pmb x}_i^j)\!=\!0,\!1,\!\ldots ,\!N-k\!\right\} \nonumber \\ \end{aligned}$$

(5)

$$\begin{aligned}&\Delta (l({\pmb x}_i^j ))=\frac{\left\| {{\pmb F}({\pmb x}_i^j ,\pmb {\alpha }^k)-\hat{\pmb F}({\pmb x}_i^j,\pmb {\alpha }^{k+(l({\pmb x}_i^j))})} \right\| }{\left\| {{\pmb F}({\pmb x}_i^j ,\pmb {\alpha }^k)} \right\| } \end{aligned}$$

(6)

where $\Delta (l({\pmb x}_i^j)$ represents the relative fitness difference under the adjacent dynamic environment about an individual ${\pmb x}_i^j$. Here, ${\pmb x}_i^j$ comes from the robust Pareto solutions RPS(i). The smaller the difference is, the solution can adapt to the new environment better. $\bar{l}({\pmb x}_i^j)$ is the maximum survival time for ${\pmb x}_i^j$ to meet the fitness threshold. The survival time of each solution in RPS(i) should be larger or equal to $L_i$.

Taking the robustness as the objective, the main algorithm steps of RPOOT-based dynamic multi-objective evolutionary optimization method are listed in Algorithm 2.

Apparently, both TMO and RPOOT are effective problem-solvers for DMOPs, which provide the Pareto-optimal solutions with different convergence and switching cost. How to choose the most suited method in the presence of a certain environmental change is still an open and challenging issue.

Indexes measuring environmental changes

In DMOPs, the environmental parameters change over time with various intervals and increments. To quantitatively measure the changes, two indexes, i.e., the frequency and intensity of environmental changes, are presented. Most existing research focuses on DMOPs where the environment changes with the same interval or intensity. However, the dynamism factors in practical applications may occur irregularly, which are described by the environmental parameters that change over time aperiodically, leading to difficulties in converging to the true Pareto front as soon as possible under each environment.

The frequency and intensity of environmental changes

The frequency of environmental changes, denoted as $f^k$, reflects how many new environments occur within a given period of time. To be specific, $f^k=\frac{1}{\tau ^k}$ and the environmental parameters remain unchanged under the kth environment. A smaller $f^k$ means that the dynamic factors have an occasional influence on the optimization problems. So far, only static $f^k$ has been considered in most dynamic optimization problems. However, the environment in practical applications may change with variable intervals. Based on this, the frequencies of environmental changes are expressed by $(f^1, f^2, \ldots , f^{N})$, and $f^k$ may differ from $f^{k+1}$ depending on the actual situation.

The environmental parameter, $\pmb {\alpha }^k$, has a direct impact on the landscape of the optimizaiton problem under the kth environment. Assume that the increment of the parameter under two adjacent environments is denoted as $\Delta {\pmb {\alpha }^{k+1}}=\pmb {\alpha }^{k+1}-\pmb {\alpha }^k$. For various DMOPs or a DMOP under different environments, the same increment may generate distinct landscapes due to the nonlinear relationship of $\pmb {\alpha }^k$ to $F(\pmb x,\pmb {\alpha }^t)$. In order to accurately measure the influence of changing environmental parameters over time on the landscape, the intensity of environmental changes is defined as follows:

$$\begin{aligned} s^{k+1}=\frac{1}{N_{P^{k}}}\sum \limits _{\pmb {x}\in {P^k}}\frac{\left\| {{\pmb F}(\pmb x ,\pmb {\alpha }^k)-{\pmb F}(\pmb x, {\pmb {\alpha }}^{k+1})} \right\| }{\left\| {{\pmb F}(\pmb x ,\pmb {\alpha }^k)}\right\| }. \end{aligned}$$

(7)

Let $N_{P^k}$ be the population size. A larger $s^{k+1}$ means that the landscape may significantly vary with the environmental parameter in the adjacent environments. Under these circumstances, the robust Pareto-optimal solutions are difficult to get satisfactory convergence in the adjacent environments, thus, RPOOT is not a wise choice. Thus, it is vital to properly estimate the intensity of changes of the future environments, which determines whether the TMO or RPOOT approach should be adopted.

Fig. 1

The true Pareto fronts of FDA3 under three adjacent environments

Taking a benchmark function, FDA3 [27], as an example, the true Pareto fronts under three adjacent environments are shown in Fig. 1. All environmental parameters change over time with the same increment and $\Delta {\pmb {\alpha }^{k+1}}$ is set to 0.1 or 0.2, respectively. Under the same $\Delta {\pmb {\alpha }^{k+1}}$, three Pareto fronts corresponding to 1st, 2nd and 3rd environments are parallel, however, the intensity of the latter environmental changes is clearly larger than that of the former one. By comparing the Pareto fronts shown in Fig. 1a, b, the environmental parameter changes with a larger increment result in the more significant shift of the landscape, and the intensity of the environmental changes is more severe. Apparently, the Pareto-optimal solution having the acceptable convergence under the dynamic environments shown in Fig. 1a can be found by RPOOT because $s^2$ and $s^3$ are both smaller than $\eta =0.4$. By contrast, RPOOT can not obtain the satisfactory Pareto-optimal solutions in any environment shown in Fig. 1b due to the larger intensity. Hence, the environmental parameters directly influence the variation of the landscape in various environments.

To predict the environmental parameters in the future

For a DMOP with time-varying environmental parameters, the frequency and intensity of environmental changes determine which of the two approaches should be employed as a more effective one in future environments. To this end, it is necessary to accurately estimate the above-mentioned characteristics of the environments that are changing.

Both of the characteristics are related to $\pmb {\alpha }^k$. In particular, the intensity of the $(k+1)$th environmental change can be estimated by Eq. (6) based on the environmental parameters in the next environment, expressed by $\pmb {\alpha }^{k+1}$. Hence, it is assumed that the future environmental parameters can be predicted. Until now, most studies assume that the environmental parameters linearly change over time [11, 27]. However, in the actual DMOPs, the environmental parameters may change periodically, randomly, or chaotically.

To reliably predict the environmental parameters in the future environments, Autoregressive Integrated Moving Average Model (ARIMA) [39], a widely-used time series prediction method, is employed. Assume that the time-varying environmental parameters form a time series, expressed by $\{\pmb {\alpha }^1,\pmb {\alpha }^2,\ldots ,\pmb {\alpha }^k\}$. After smoothing it to a stationary one, the autoregressive, the moving average or the autoregressive moving average is selected to build the ARIMA(p, d, q) model, which is then used to forecast the future environmental parameters. The detailed modeling process is illustrated as follows.

• (1) Judge the stationarity of the time series $\{\pmb {\alpha }^k\}$ by the Augmented Dickey–Fuller (ADF) Test [39].

• (2) If $\{\pmb {\alpha }^k\}$ is non-stationary, the following smoothing process is iterated until a stable time series, denoted as $\{\nabla ^n \pmb {\alpha }^k\}$, is obtained. Let d be the differential times.

  $$\begin{aligned} {\text {1-order difference:}}&\nabla \pmb {\alpha }^k=\pmb {\alpha }^{k+1}-\pmb {\alpha }^k \end{aligned}$$

  (8a)
  $$\begin{aligned} {\text {n-order difference:}}&\nabla ^n \pmb {\alpha }^k=\nabla ^{n-1} \pmb {\alpha }^{k+1}-\nabla ^{n-1} \pmb {\alpha }^k \end{aligned}$$

  (8b)

• (3) Given $\{\nabla ^n \pmb {\alpha }^k\}$, if its partial correlation function has the truncation property and the autocorrelation function has the trailing edge, AR(p) is employed to build the prediction model, and p is the autoregressive term. By contrast, MA(q) is suitable for $\{\nabla ^n \pmb {\alpha }^k\}$, which has a trailing edge of its partial correlation function and the truncation of the autocorrelation function. q is the moving average term. Once both the partial correlation and autocorrelation functions have a trailing edge, ARMA(p, q) is used as the prediction model.

• (4) The environmental parameters in the future environments are predicted by the above ARIMA(p, d, q) model.

Proposed algorithm

Once the environmental parameters of a DMOP have changed, it is necessary to select one of the two approaches to dynamic multi-objective optimization method according to the predicted characteristics of environmental changes, with the purpose of finding the Pareto-optima or robust ones that meet the requirements of decision makers. To this end, a hybrid dynamic multi-objective evolutionary optimization algorithm is proposed.

Definition of switching cost

Neither the TMO-based nor RPOOT-based optimization approaches take into account switching costs, the former switches to a new solution at every dynamic moment, while the latter switches to the new solutions only when the solution does not satisfy robustness. In the real world dynamic optimization problems, such as production scheduling and path planning [1, 4], an optimal solution found in the new environment may remarkably differ from the historical one, resulting in the huge cost for adjusting the related human or resources. In this situation, the newly-obtained optimal solution is unavailable due to the unacceptable costs incurred in switching solution.

To get an executable optimal solution, Huang et al. [21] took switching cost as an objective, and presented a multi-objective optimization framework for robust optimization over time. Furthermore, ROOT/SCII [40] is proposed to simultaneously maximize the robustness and minimize the switching cost, and select a solution from the obtained Pareto set to be used in the new environment. Yazdani [41, 42] considered the time-linkage characteristic of dynamic optimization problems, and proposed a new semi robust optimization over time, with the purpose of balancing switching cost and the quality of solutions. Based on the above-mentioned switching cost presented for dynamic single-objective optimization problems, we define the improved switching cost for the Pareto-optimal solutions from the kth to the $(k+1)$th dynamic environment, denoted as $sc^{k+1}$, as follows.

$$\begin{aligned} sc^{k+1}=\frac{1}{N_{P^k}}\sum \limits _{\pmb {x}\in {P^k}}\min \limits _{\pmb {x'}\in {P^{k+1}}}\frac{\left\| {\pmb x-\pmb {x'}} \right\| }{\left\| {\pmb x}\right\| } \end{aligned}$$

(9)

In above formula, $P^k$ represents the Pareto-optimal solution set in the kth environment, and $N_{P^k}$ is its size. ${\pmb x}'$ is an Pareto-optimum in the $(k+1)$th environment. $sc^{k+1}$ represents the average relative error of the Pareto solutions switching from the kth environment to the $(k+1)$th environment. The switching cost of solutions can provide decision makers the information for determining whether the solution should be switched or not.

Selection criteria

TMO and RPOOT are two different problem-solvers for DMOPs. In TMO, the optimization process will be triggered whenever an environmental change is detected. If the environments vary with a short interval, there is insufficient time for the population to converge to the true Pareto front. In this case, RPOOT can provide feasible and satisfactory solutions derived from previous environments without finding new ones in the current environment. Although no cost will be spent on tracking the new Pareto-optimal solutions, the robust ones found by RPOOT are just acceptable suboptimal in the subsequent environments. More specifically, the Pareto front of the historical Pareto-optimal solutions may be far away from the true one in the current environment, especially for DMOPs with strong environmental changes. In this instance, TMO will be employed instead of RPOOT, with the purpose of obtaining new optimal solutions. Therefore, the dynamic characteristics of environmental changes and switching cost are both key factors that determine which dynamic multi-objective optimization method should be adopted in the next environment.

Suppose that $c^k_f$, $c^k_s$ and $c^k_{sc}$ are the coefficients for the frequency and intensity of environmental changes, as well as switching cost, respectively. ${\delta }_f$, ${\delta }_s$ and ${\delta }_{sc}$ are the corresponding thresholds preset by a decision maker.

$$\begin{aligned}&c^k_f=\left\{ \begin{array}{ll} 1 &{} \; f^k > {\delta }_f\\ 0 &{} \; f^k \le {\delta }_f \end{array} \right. \end{aligned}$$

(10)

$$\begin{aligned}&c^k_s=\left\{ \begin{array}{ll} 1 &{} \; s^k \le {\delta }_s\\ 0 &{} \; s^k > {\delta }_s \end{array} \right. \end{aligned}$$

(11)

$$\begin{aligned}&c^k_{sc}=\left\{ \begin{array}{ll} 1 &{} \; sc^k > {\delta }_{sc}\\ 0 &{} \; sc^k \le {\delta }_{sc} \end{array} \right. \end{aligned}$$

(12)

$c^k_f=1$ and $c^k_s=1$ represent that the environment changes significantly within a short time period and low intensities. Similarly, $c^k_{sc}=1$ indicates that switching cost of a solution is too high to be accepted by decision makers.

As we known, RPOOT-based dynamic robust multi-objective evolutionary optimization method is suited for solving DMOPs with frequent, but weak environmental changes, namely, $c^k_f=1$ and $c^k_s=1$. In this case, tracking the Pareto-optima in the current environment during the limited time is difficult. On the contrary, if the environments vary with a large interval or strong intensity, that is, $c^k_f=0$ or $c^k_s=0$, DMOP in each environment can be regarded as a static MOP. Thus, TMO-based dynamic multi-objective evolutionary optimization method has enough time to find the Pareto-optimal solutions approximating to the true one. Furthermore, if the cost of switching the solutions to be implemented in two consecutive environments is relatively low, the obtained new Pareto-optimal solutions are acceptable. Otherwise, the sub-optimal Pareto solutions approximating to the optimal ones in the last environment will be employed. Based on this, the criteria for choosing dynamic multi-objective optimization methods are presented as follows.

Selection criteria In the kth environment,

• (1) If $c^k_s=0$, TMO-based dynamic multi-objective evolutionary optimization method is employed.

• (2) if $c^k_s=1$ and $c^k_f=1$, RPOOT-based dynamic robust multi-objective evolutionary optimization method is adopted.

• (3) if $c^k_s=1$ and $c^k_f=0$, TMO-based dynamic multi-objective evolutionary optimization method is employed.

  • (a) if $c^k_{sc}=1$, the sub-optimal solution close to the optimal ones obtained in the last environment is used in the current problem.

  • (b) if $c^k_{sc}=0$, the new Pareto-optimal solutions are employed.

Overall framework

The pseudo-code of hybrid dynamic multi-objective evolutionary optimization algorithm (HDMEOA) is shown in Algorithm 3. The key issues are illustrated one by one as follows.

• (1) Construct the time series of environmental parameters In the kth environment, the time series of the environmental parameters denoted as $\{\pmb {\alpha }^1,\pmb {\alpha }^2,\ldots ,\pmb {\alpha }^{k}\}$ are formed by the historical ones. The corresponding frequencies of environmental changes are $(f^1, f^2, \ldots , f^{k})$.

• (2) Predict the environmental parameters Based on $\{\pmb {\alpha }^1,\pmb {\alpha }^2,\ldots ,\pmb {\alpha }^{k}\}$, the ARIMA prediction model is built and employed to estimate the future environmental parameters $\hat{\pmb {\alpha }}^{k+1}, \ldots , \hat{\pmb {\alpha }}^{k+T-1}$. Following that, the fitness values of an individual, expressed by $F(\pmb {x},\hat{\pmb {\alpha }}^{k+1}),\ldots , F(\pmb {x},\hat{\pmb {\alpha }}^{k+T-1})$, are calculated.

• (3) Choose the evolutionary optimization method The intensity of environmental changes is estimated in terms of predictive fitness values. Based on the estimated dynamic characteristics and switching cost, TMO- or RPOOT-based dynamic multi-objective evolutionary method is selected to obtain satisfactory Pareto solutions in the current environment according to the designed criteria.

Table 1 Benchmark functions

Table 2 The main parameters of the HDMOEA

Experimental study

To verify the effectiveness of the proposed HDMEOA framework, three groups of experiments are designed. The effectiveness of the prediction model is first analyzed in the presence of linear, chaotic and stochastic environmental changes. Following that, the performance of HDMEOA is further compared with TMO- and RPOOT-based dynamic multi-objective evolutionary optimization methods, as shown in Algorithms 1 and 2, respectively. Three kinds of methods all employ multi-objective evolutionary algorithm based on decomposition (MOEA/D) [43] as the optimizer. The t-test is carried out to indicate significance between different results at the 0.05 significance level [29, 44]. If there is a significant difference between the two, the observation is labeled as ‘$+$’ after the value. Otherwise, there is a ‘−’ after the value. All experiments are carried out on a PC with Intel Core i5-2430M CPU 2.4 GHz, 16 GB RAM, Windows 10 operating system, and Matlab R2016a.

Benchmark functions and performance metrics

All experiments are performed on eight commonly-used dynamic benchmark functions, including FDA1-FDA5 [27] and DMOP1-DMOP3 [11]. Their definition and characteristics are presented in Table 1, and the main parameters of HDMEOA proposed in this work are listed in Table 2.

To synthetically evaluate the convergence and robustness of the Pareto-optimal solutions obtained by TMO and the robust ones found by RPOOT, the mean robust survival time (MRST) and mean robust inverse generation distance (MRIGD) defined by Chen et al. [24] are introduced as the metrics.

Mean robust survival time The MRST of the Pareto-optimal solutions obtained by TMO or RPOOT under all environments reflects their adaptability to the dynamic environments. Let $L_i$ be the survival time of the ith Pareto-optimal solution. In particular, $L_i=1$ means that the Pareto-optimal solution was obtained by TMO.

$$\begin{aligned} \text{ MRST }=\frac{1}{L} \sum _{i=1}^{L}L_i \end{aligned}$$

(13)

Mean robust inverse generation distance The MRIGD comprehensively measures the convergence and distribution of the Pareto-optimal solutions within their survival time.

$$\begin{aligned} \text{ MRIGD } =\frac{1}{L} \sum _{i=1}^{L}\mathop {\max }\limits _{q=k_{i} ,\ldots ,k_{i} +L_{k_{i} } } \text{ IGD }(q) \end{aligned}$$

(14)

Prediction accuracy

Environment-driven HDMEOA provides a generic framework for various problem-solvers of DMOPs, with the purpose of obtaining the Pareto-optima that meet the requirements of a decision maker under a given environment. To fully verify the effectiveness of the proposed framework, three types of environmental changes that commonly appear in actual DMOPs are taken into account, in which the environmental parameters change over time with the linear [11, 27], chaotic and Stochastic [30] mode as shown in Fig. 2.

Fig. 2

The dynamic environmental parameters

Fig. 3

Comparison of the estimated environmental parameters

Linear change Assume that the dynamic environmental parameters start from 0, and increase linearly with a fixed increment of 0.2 [11, 26]. Since $N=50$, the range of ${\alpha }^k\in [0,10]$, as shown in Fig. 2a.

Chaotic change Chaos is derived from nonlinear dynamical systems, whose trajectory is determined by the initial value and the chaotic mapping parameters [29] as follows.

$$\begin{aligned} x_{k+1}=\mu x_{k}(1-x_{k}),\;\;\mu \in [0,4],x_k\in (0,1). \end{aligned}$$

(15)

In the experiment, the logistic parameter $\mu =4$, and the initial value $x_0=0.6$. Following (15), the environmental parameter $\alpha ^k=10x_k$, and ${\alpha }^k\in [0,10]$, as shown in Fig. 2b.

Stochastic change The environmental parameters is generated by a random number satisfying uniform distribution in [0,10], as shown in Fig. 2c.

Fig. 4

IGD comparison for each time among three methods under linear environmental changes

For the above three environmental changes, the future environmental parameters are estimated by the ARIMA-based prediction model. As shown in Fig. 3, under a dynamic time step $k = 19$, the real and predicted fitness values are compared in the future T adjacent environments. Apparently, the prediction performance under linear change is the best, and the mean square deviation is 8.8818e−16. The prediction results under chaos and stochastic change is similar, and their mean square deviations are 0.8258 and 0.7053, respectively. Thus, predicting the range of chaos and stochastic change is more feasible.

Performance comparison under linear environmental changes

Suppose that the environmental parameters $\alpha ^k\in [0,10], k=1,2,\ldots ,N$ linearly increase with the fixed increment about 0.2. Here, $N=50$ and $\alpha ^0=0$. Under the above environments, the proposed algorithm is compared with TMO- and RPOOT-based dynamic multi-objective evolutionary optimization methods. Figures 4, 5 and 6 depict the IGD and survival time indicators obtained by different algorithms parameters. Tables 3, 4 and 5 list the corresponding experimental results, where data are the average and standard deviation, the boldface ones are the best values.

Fig. 5

Survival time comparison of three methods under linear environmental changes

Fig. 6

Survival time comparison of HDMEOA with various $\delta _s$ under linear environmental changes

By comparing the MRIGD of the three algorithms listed in Table 3, and the IGD of each time in Fig. 4, we can see that TMO-based dynamic multi-objective evolutionary optimization method has the smaller MRIGD than the others in most cases, which indicates that the corresponding Pareto-optimal solutions are the closest to the true Pareto front. By contrast, the robust Pareto-optimal solutions achieved by RPOOT has the poorest convergence performance due to their acceptable suboptimal performance in the adjacent environments. In addition, the MRST values listed in Table 4 indicate that the mean survival time of the robust Pareto solutions achieved by RPOOT is much larger than that of TMO, meaning that RPOOT achieves acceptable solutions in more environments, is quicker in finding optimal solutions in a new environment, and incurs less switching cost than TMO. Apparently, neither of them is able to balance the quality and robustness of solutions.

Different from TMO and RPOOT, HDMEOA has achieved similar MRIGD values to TMO while achieving fewer robust Pareto-optimal solutions than RPOOT except for Fun2. Taking Fun7 as an example, the Pareto-optimal solutions obtained by HDMEOA and TMO have a similar MRIGD value (around 0.063), which is smaller than 0.1551 obtained by RPOOT. In particular, the proposed algorithm has the best MRIGD for Fun4 and Fun5, indicating the best convergence. Moreover, the number of environments in which TMO or RPOOT is adopted, respectively, in HDMEOA listed in Table 5 shows that TMO is employed in 44 out of 50 environments, and 2 robust Pareto-optimal solutions are used in other environments. The mean robust survival time of HDMEOA is 1.064, which is shorter than that of RPOOT. Less robust Pareto-optima over time promotes the quality of the solution set, but weakens their robustness.

To further compare the robustness of the three methods, Fig. 5 depicts the robust survival time of all Pareto-optimal solutions obtained in each environment as the environmental parameters change linearly. Apparently, TMO-based multi-objective evolutionary optimization method needs to track each true Pareto front. Thus, the survival time of all solutions is 1. By contrast, the robust candidates found by HDMEOA and RPOOT remain in use under more than one environment, consequently having the survival time larger than 1. More specifically, the robust Pareto fronts obtained by RPOOT have the longest survival time, which shows the strongest robustness.

Table 3 MRIGD comparison of three methods under linear environmental changes

Table 4 MRST comparison of three methods under linear environmental changes

Table 5 NumTR comparison of three methods under linear environmental changes

In addition, the experimental results listed in above tables indicate that both HDMEOA and RPOOT obtained worse MRIGD values with the increase of $\eta$, but their MRST values are longer than these obtained by TMO. Taking a closer look, we find that, more sub-optimal solutions are acceptable as $\eta =0.4$, resulting in the worse convergence but enhanced robustness. For HDMEOA, Fig. 6 depicts the survival time of all Pareto-optimal solutions under various threshold values for the intensity of environmental changes. As $\delta _s=0.4$, the survival time of each robust solution becomes larger, meaning better robustness, because RPOOT is triggered under more environments with severer changes of the landscape.

Performance comparison under chaotic and stochastic environmental changes

The chaotic change is more complex than the linear one, causing more intensive environmental changes. By comparing with Table 3, the experimental results listed in Table 6 show that the obtained Pareto-optimal solutions have worse convergence under the environment with chaotic changes. As can be seen from Table 7, the survival time of HDMEOA is slightly shorter than that of RPOOT, but longer than that of TMO. In HDMEOA, $c^k_s=0$ in most of the environments, and the TMO-based multi-objective evolutionary optimization method is employed according to the proposed criteria, as shown in Table 8. Moreover, HDMEOA has achieved similar MRIGD values to those achieved by TMO except on Fun6, where both HDMEOA and TMO are both significantly better than RPOOT, indicating that the Pareto-optimal solutions obtained by HDMEOA and TMO are closer to the true Pareto front.

Table 6 MRIGD comparison of three methods under under chaotic environmental changes

Table 7 MRST comparison of three methods under chaotic environmental changes

Table 8 NumTR comparison of three methods under chaotic environmental changes

Table 9 RIGD comparison of three methods under stochastic environmental changes

Table 10 MRST comparison of three methods under stochastic environmental changes

Table 11 NumTR comparison of three methods under stochastic environmental changes

Under the adjacent environments randomly change within a certain range, the intensity of environmental change may not satisfy the preset threshold, and then RPOOT is seldom triggered. As shown in Tables 9, 10 and 11, for all benchmarks except for Functions 2 and 4, TMO-based dynamic multi-objective evolutionary optimization method is adopted in most dynamic time steps, but it is difficult to find the robust optimal solutions. The survival time of HDMEOA as shown in Table 10 is shorter than that of RPOOT, but longer than TMO. Also, we observe from Table 9 that MRIGD of the solution sets gotten from HDMEOA more approximate to that of TMO.

As shown in Tables 3, 6 and 10, the MRIGD values of the non-dominated solutions found by RPOOT are much larger than those of the other two methods, implying the poor convergence. HDMEOA has much smaller MRIGD values and exhibits the simliar performance to TMO due to the proposed selection criteria, especially on Fun6. The Pareto sets of Fun2 and Fun4 do not change, while the corresponding true Pareto fronts vary with environmental changes. As $\delta _s=0.4$, the number of robust Pareto fronts obtained by HDMEOA and RPOOT are almost the same and their mean robust survival times are longer, RPOOT is better suited for solving DMOPs having the similar characteristics as the above functions.

Conclusion

Traditional TMO-based dynamic multi-objective evolutionary optimization methods may not be able to satisfactory solutions with small switching cost in the presence of frequent and severe environmental changes. In contrast to TMO, RPOOT-based dynamic robust multi-objective evolutionary optimization methods are developed to find robust Pareto-optima whose performance is acceptable in a number of adjacent environments. However, both methods can not balance the quality of solutions and the cost for switching and computation. To address the above dilemma, an environment-driven HDMEOA has been proposed in this paper. Firstly, two indexes, including the frequency and intensity of environmental changes are defined. Secondly, a criterion is presented based on the characteristic of dynamic environments and switching cost of solutions, with the purpose of selecting an appropriate method from TMO and RPOOT in each environment. To test the effectiveness of the proposed strategy, the experiments are designed based on eight benchmark functions under linear, chaotic and stochastic environmental changes. The statistical results reveal that the HDMEOA can effectively choose the more suited method (TMO or RPOOT) that meets the requirements of decision makers. It is very meaningful to strike a good balance between the convergence and robustness of the Pareto-optimal solutions.

It is worth noting that only linear ,chaotic and stochastic environmental changes are considered in this paper, however, there exist other types of dynamic environmental changes in practice. It is interesting to study the influences and effects of the HDMEOA on real application dynamic optimal problems. Furthermore, predicting environmental changes accurately plays a key role in choosing the suited optimization methods from TMO and RPOOT. As we known, any prediction method has the different accuracy for dynamic problems with various kinds of environmental changes. For example, the ARIMA model used in this paper has a good prediction effect on linear change, but not on chaotic and stochastic changes. To solve this drawback, one may consider more prediction methods, such as [26, 44, 45], to track the dynamic environments with other nonlinear changes.