Abstract
Dynamic multiobjective optimization problems (DMOPs) challenge multiobjective evolutionary algorithms (MOEAs) because of the varying Paretooptimal sets (POS) over time. Research on DMOPs has attracted a great interest from academic, due to widespread applications of DMOPs. Recently, a few learningbased approaches have been proposed to predict new solutions in the following environments as an initial population for a multiobjective evolutionary algorithm. In this paper, we propose an alternative learningbased method for DMOPs, a deep multilayer perceptronbased predictor to generate an initial population for the MOEA in the new environment. The historical optimal solutions are used to train a deep multilayer perceptron which then predicts a new set of solutions as the initial population in the new environment. The deep multilayer perceptron is incorporated with the multiobjective evolutionary algorithm based on decomposition to solve DMOPs. Empirical results demonstrate that our proposed algorithm is effective in tracking varying solutions over time and shows great superiority comparing with stateoftheart methods.
Introduction
Evolutionary dynamic multiobjective optimization (EDMO) has been studied for a few years [1,2,3]. Unlike evolutionary multiobject optimization [4], an EDMO algorithm needs to track the timevarying Paretooptimal solutions (POS)/Paretooptimal front (POF) as they change over time [5, 6]. Many realworld multiobjective optimization problems are dynamic in nature, e.g., job shop scheduling [7], online optimization of shift strategy for hydromechanical continuously variable transmission [8], control of timevarying unstable plants [9]. Evolutionary algorithms (EAs) are still challenged in solving EDMO due to their convergence nature and the problems’ changes [10, 11]. It is hard to track the timevarying POS or POF for a converged population, due to losing exploration ability [12, 13]. A typical strategy is to treat as a new optimization problem when POS or POF changes, whereas reinitializing the converged population can recover EAs’ exploration ability [5]. However, reinitializing the population does not make full use of the historical information about exploration, it may not locate the new POS and POF before changing again. In many realworld dynamic multiobjective optimization problems (DMOPs), the objective functions may change over time regularly, rather than randomly [14, 15]. There may be some correlations between POS or POF between several consecutive time steps. Therefore, predicting new solutions according to the historical optimal solutions has been widely used for solving DMOPs [16,17,18].
A few prediction models have been proposed to assist EAs for solving DMOPs whose environments change regularly. Considering the determination of parameters in the prediction models, these methods can be loosely categorized into two types: nonlearning based and learning based. The nonlearning based methods design a mathematical model whose free parameters are determined experimentally, theoretically or by experience. Linear model, kalman filter [16], differential prediction [19] belong to this kind of method. Zhou et al. [5] proposed a simple linear model to predict a new population which were then perturbed by a Gaussian noise whose variance was estimated according to previous changes. The linear model only used two previous solutions to compute the moving distance of the adjacent two time steps, and assumed that the solution move in the same distance. Muruganantham et al. [16] proposed to use a Kalman Filter (KF) to track the movement of each solution. In the KF model, the state of a process represented solutions in the time step, and a statetransition matrix represented the motion of solutions. Cao et al. [19, 30] proposed a differential model to predict the movement of centroid using its locations in three previous environments. Instead, the learning based methods determine the free parameters by learning from the historical data, which are implemented through some machine learning methods, e.g., linear regression [17], support vector machine [21, 22], transfer learning [23,24,25]. Hatzakis and Wallace [6] proposed a forwardlooking approach to predict new solutions in new environments, where an autoregressive (AR) model was used for forecasting. Cao et al. [22] proposed to use support vector regression (SVR) as the predictor to generate the initial population in each new environment. Jiang et al. [23] used a transfer learning technique to create the initial population via reusing past knowledge. They assumed that the distributions of solutions for a DMOP in different environments are different but correlated, the different distributions could be mapped into a latent space wherein they were similar.
Most of proposed prediction methods in literature are designed according to some prior knowledge or assumption, e.g., solutions at several successive time steps are linearly correlated. However, it is unknown about the correlations of solutions at different time steps before solving. A linear prediction model can only cover a limited range of DMOPs, it may perform badly if solutions at different time steps are nonlinear correlated or have more complex correlations.
Instead of assuming some specific correlations of historical solutions in DMOPs, we regard the prediction model as a blackbox, no matter what the correlations of historical solutions. Recently, artificial neural networks (ANNs) [26] have gained increasing attention again, since of development of computing power and large scale of data. ANNs theoretically are able to fit any functions among the given data, which could be employed to learn the correlations of historical optimal solutions in solving DMOPs. In addition, ANNs have strong robustness, which could tolerate noise existing in historical optimal solutions.
In this paper, to solve DMOPs, we adopt a branch of ANNs, called multilayer perceptron (MLP), to construct a prediction model combined with a multiobjective evolutionary algorithm based on Decomposition (MOEA/D) [27, 28]. The motivation is that we assume the solutions obtained in consecutive time steps are autocorrelated. In each environment, MOEA/D decomposes a multiobjective optimization problem into several scalar optimization subproblems and optimizers them simultaneously. Then each subproblem obtains its corresponding solutions in different environments to construct a sequence of solutions, which are used to train the MLPbased predictor. The trained predictor then predicts a novel solution in later new environment, which is placed into an initial population for optimizing a novel multiobjective optimization problem in later new environment. The experiments show that our proposed multilayer perceptronbased multiobjective evolutionary algorithm based on decomposition (MOEA/DMLP) is effective in tracking dynamic POS/POF, and outperforms the stateoftheart methods on most of benchmarks.
The remainder of this paper is organized as follows. Section “Background” introduces the background and related work of EDMO. Section “Related works” gives a brief introduction of ANNs and details the proposed algorithm: MOEA/DMLP. In Sect. “Proposed method”, we design the experiments and present the experimental results in Sect. “Experimental design”. Section “Result and discussion” gives the conclusion.
Background
Definition of DMOPs
In this paper, we focus on the following dynamic multiobjective optimization problems:
where t represents the discrete time instants, \({\varvec{x}}\) is the decision vector, m is the number of objectives, and \(\Omega \) represents the decision space. \({\varvec{F}}\left({\varvec{x}},t\right)\) is composed of m timevarying objective functions.
Definition 1 (Pareto solution) At time t, a solution \({{\varvec{x}}}_{1}\in\Omega \) Pareto dominates another solution \({{\varvec{x}}}_{2}\in\Omega \), denoted by \({{\varvec{x}}}_{1}\succ {{\varvec{x}}}_{2}\), if and only if:
Definition 2 (Paretooptimal set) If a solution \({{\varvec{x}}}^{*}\in\Omega \) is said to be nondominated if and only if there is no other solution \({\varvec{x}}\in\Omega \) such that \({\varvec{x}}\succ {{\varvec{x}}}^{*}\). The Paretooptimal set (POS) is the set of all Paretooptimal solutions, that is:
Definition 3 (Paretooptimal front) At time t, the Paretooptimal front (POF) is the corresponding objective vectors of the POS:
An EDMO algorithm is required to obtain a set of solutions as closely as possible to the POF at any discrete time instant.
Related works
A variety of predictionbased approaches have been proposed to solve DMOPs, which can be loosely categorized into two types: nonlearning based and learning based.
Nonlearning based: Zhou et al. [5] proposed a simple linear model to predict a new population which were then perturbed by a Gaussian noise whose variance was estimated according to previous changes. The linear model only used two previous solutions to compute the moving distance of the adjacent two time steps, and assumed that the solution move in the same distance. This simple strategy has been employed by many algorithms to associate with other ideas to solve DMOPs. Wu et al. [20] proposed to integrate a local search strategy with the linear model to generate an initial population. Instead of computing each solution’s moving distance, Zou et al. [18] proposed to use the center point and knee point to compute the moving distance, and new solutions were predicted based on the movement of center point. Ruan et al. [29] also proposed to use the motion direction of center points, but without involving noise. Jiang and Yang [11] used this linear model to predict the motion direction and movement stepsize of centroids of population. Instead of using two former solutions, Cao et al. [19, 30] proposed a differential model to predict the movement of centroid using its locations in three previous environments. Muruganantham et al. [16] proposed to use a Kalman Filter (KF) to track the movement of each solution. In the KF model, the state of a process represented solutions in the time step, and a statetransition matrix represented the motion of solutions. They developed a 2D KF and a 3D KF model to describe the movement. In the 2D KF, the satetransition was the same with the above linear model, and a secondorder linear model was used in the 3D KF pattern. Rambabu et al. [31] proposed a mixtureofexpertsbased ensemble framework to solve DMOPs, in which the framework utilized multiple prediction mechanisms including two variants of KF and a linear model. A gating network was applied to switch these prediction models based on the performance of the predictors at different time steps. Rong et al. [32] proposed a multimodel prediction method for DMOPs, wherein they defined four types of the Pareto Solution change and a method of determining the type of change. For different types of change, they provided different prediction models, e.g., the linear movement of the centroids of solutions for translation change type, the linear movement of the cluster for rotation change type. In addition, Rong et al. [33] presented a multidirectional prediction strategy to enhance the performance of algorithm for DMOPs. The population was clustered into several representative groups by a proposed classification strategy, where the number of clusters was adapted according to the intensity of the environmental change. The evolutionary direction of each individual was estimated based on the movement from its location in the previous environment to the current location. Wang et al. [34] proposed a grey prediction model using the centroid point of each cluster to generate the initial population. Hu et al. [35] divided each individual into two parts: the microchanging decision and macrochanging decision based on the intensity of environmental change. Ma et al. [15] proposed to predict new locations of center points using a series of center points in different subregions based on a difference model. Wang et al. [36] proposed to use the Gaussian Mixture Model to fit various data distributions for the prediction of the new POS. Liang et al. [37] proposed to divide the decision variables into two and three different groups in static optimization and changes response stages, respectively.
Learning based: Hatzakis and Wallace [6] proposed a forwardlooking approach to predict new solutions in new environments, where an autoregressive (AR) model was used for forecasting. In the first 100 time steps, the EA algorithm was execute to collect data for training the AR model, then the fitted model could be used to predict new solutions in the later new time steps. The AR model was employed by Zhou et al. [17] in their algorithm. Instead of predicting some isolated points, the AR model tracked the centroid of solutions in the different time steps. Jiang et al. [23] used a transfer learning technique to create the initial population via reusing past knowledge. They assumed that the distributions of solutions for a DMOP in different environments are different but correlated, the different distributions could be mapped into a latent space wherein they were similar. In DMOPs, the obtained solutions in two successive environments were mapped into the latent space wherein they obeyed the same distribution. In addition, they proposed a new individual transferbased evolutionary algorithm for DMOPs [38]. The motivation is the negative transfer would guide the searching to a wrong direction. Thus they proposed to find a few highquality individuals with better diversity to avoid negative transfer caused by individual aggregation. Cao et al. [22] proposed to use support vector regression (SVR) as the predictor to generate the initial population in each new environment. Zhang et al. [39] proposed to use the centroid distance to measure the distance of the population centroid and reference points, which was combined with the transfer learning method to predict the initial population. Zou et al. [40] proposed a reinforcement learning approach for DMOPs, which relocated the individuals based on the severity degree of environmental changes. The motivation of this approach was that the reinforcement learning is effective to learn the optimal behavior by interactions between an agent and dynamic environments. Wang et al. [41] proposed an ensemble learning based prediction strategy to help algorithms reinitialize a new population for DMOPs, including linear model, knee pointbased autoregressive model, populationbased autoregressive model, and random reinitialization model. Feng et al. [42] proposed to predict the moving of POS via an autoencoding evolutionary search method.
Proposed method
MOEA/D
The algorithm MOEA/D was proposed by Zhang and Li [27] in 2007, which has been a very popular and effective approach for solving MOPs. Instead of using nondominated sorting strategy for handling multiple objective optimization problems, MOEA/D decomposes multiple objective optimization problems into a few singleobjective optimization problems by an aggregation function. In this paper, we adopt the Tchebycheff approach to decompose MOPs. Let \({{\varvec{\lambda}}}^{1},{\dots ,{\varvec{\lambda}}}^{N}\) be a set of even spread weight vectors, a multiobjective optimization problem at time t can be decomposed into N scalar optimization problems, and the ith subproblem (\(i=1,\cdots ,N\)) at time t is given by:
where \({{\varvec{z}}}^{*}={({z}_{1}^{*},\cdots ,{z}_{m}^{*})}^{T}\) is the reference vector, i.e., \({z}_{j}^{*}=min\{{f}_{j}\left({\varvec{x}},t\right),{\varvec{x}}\in\Omega \}\) (for a minimization problem) for each \(j=1,\dots ,m\). MOEA/D minimizes these N subproblems simultaneously in a single run.
In MOEA/D, a neighborhood of weight vector \({{\varvec{\lambda}}}^{{\varvec{i}}}\) is defined as a set of its several closest weight vectors in \(\left\{{{\varvec{\lambda}}}^{1},\cdots ,{{\varvec{\lambda}}}^{N}\right\}\).The neighborhood of the ith subproblem consists of all the subproblems with the weight vectors from the neighborhood of \({{\varvec{\lambda}}}^{i}\). A population of N solutions is randomly generated and each solution is randomly allocated to a particular subproblem.
Multilayer perceptron
Artificial neural networks (ANNs) are wellknown datadriven machine learning methodologies, which simulate the neural systems of human brains [43]. These networks can model any linear/nonlinear functions by fitting datasets and generalize to unseen situations, which have been widely used for solving classification and regression problems. Multilayer perceptron (MLP) is the most utilized class of ANNs, which consists of an input layer followed by a few hidden layers and an output layer [44]. Each layer consists of a few nodes, which represent neurons (processing units) with a nonlinear activation functions. Nodes between adjacent layers are connected with weighting values [45]. Figure 1 illustrates a MLP network with a single hidden layer. The output value of each node in layers is calculated as follows:
where \({{\varvec{I}}}_{i}\) is the input vector,\({{\varvec{W}}}_{i}^{l}\) is the connection weights between \({{\varvec{I}}}_{i}\) and the node l; m is the number of input vectors, j is the number of nodes in the layer, \({\beta }_{l}\) is the bias of the l^{th} node, and \(\varphi \) is an activation function, e.g., the standard logistic sigmoid function, that is:
The use of MLPs can be divided into two phases, training and inference. In training phase, a set of training samples is used to determine the weights and bias values of MLPs. In inference phase, a trained MLP model output the result according to the input value. Training the MLP is performed by modifying the weights and bias in successive iterations, such that the error is minimized. The objective of the training is to minimize the mean of squares of the network errors (MSE) [46]:
where \({y}_{i}^{*}\) and \({y}_{i}\) are the target output and predicted output, respectively, for ith training iteration and M is the total number of training iterations.
A MLPbased predictor
We build a MLPbased predictor to assist MOEA/D in searching the new Paretooptimal solutions in the new environment. The predictor learns from the historical optimal solutions and predicts a new set of solutions, which acts as the initial population for MOEA/D. Assuming the historical optimal solutions in the previous environments are denoted as \({\{{\varvec{P}}}_{1},{{\varvec{P}}}_{2},\cdots ,{{\varvec{P}}}_{t}\}\), which are expected to be learned to estimate a new set of solutions \({{\varvec{P}}}_{t+1}\) in the later environment.
Supposing there are N solutions in each set \({{\varvec{P}}}_{i}(i=1,\cdots ,t)\), where each solution is the obtained optimal result of the corresponding subproblem in an environment. In MOEA/D, each solution is associated with its subproblem determined by the weight vector. The weight vectors are generated in the initialization stage, which remains unchanged. Therefore, the solutions, as long as obtained in different environments associated with the same weight vector, can comprise a timeseries solutions automatically. Therefore, the historical optimal solutions sets \({\{{\varvec{P}}}_{1},{{\varvec{P}}}_{2},\cdots ,{{\varvec{P}}}_{t}\}\) can comprise N time series, i.e.\({{({\varvec{x}}}_{i}^{1},{{\varvec{x}}}_{i}^{2},\cdots ,{{\varvec{x}}}_{i}^{t})}^{T},{{\varvec{x}}}_{i}^{1}\in {{\varvec{P}}}_{1},{\boldsymbol{ }{\varvec{x}}}_{i}^{2}\in {{\varvec{P}}}_{2},\cdots ,{{\varvec{x}}}_{i}^{t}\in {{\varvec{P}}}_{t}, i=1, \cdots , N.\) We assume that each solution has d variables which are independent each other, i.e. \({{\varvec{v}}}_{i}^{k}=\left({x}_{i,1}^{k},{x}_{i,2}^{k},\cdots ,{x}_{i,d}^{k}\right), k=\mathrm{1,2},\cdots ,t, i=1, \cdots ,N\). Thus, each time series \({{({\varvec{x}}}_{i}^{1},{{\varvec{x}}}_{i}^{2},\cdots ,{{\varvec{x}}}_{i}^{t})}^{T}\) can be further divided into d series of variables,\({(x}_{i,j}^{1},{x}_{i,j}^{2},\cdots ,{x}_{i,j}^{t}),i=\mathrm{1,2},\cdots ,N,j=\mathrm{1,2},\cdots ,d\), which means that we need build N*d individual prediction models to estimate a new set of solutions. Considering different variables have different correlations in time series, thus each model should be trained separately.
For each sequence of a solution in jth dimension, \({(x}_{i,j}^{1},{x}_{i,j}^{2},\cdots ,{x}_{i,j}^{t}),i=\mathrm{1,2},\cdots ,N,j=\mathrm{1,2},\cdots ,d\), we believe that there exists a hidden function to describe the correlation of the sequence. We assume each value is strongly correlated with s preceding ones, thus there is:
where \({f}_{MLP}\) denotes the MLPbased predictor.
For the sequence of the solution, \({(x}_{i,j}^{1},{x}_{i,j}^{2},\cdots ,{x}_{i,j}^{t}),i=\mathrm{1,2},\cdots ,N,j=\mathrm{1,2},\cdots ,d\), we can get (ts) training samples like \(\left\{\left(\left({x}_{i,j}^{1},\cdots ,{x}_{i,j}^{s}\right),{x}_{i,j}^{s+1}\right),\cdots ,\left(\left({x}_{i,j}^{ts},\cdots ,{x}_{i,j}^{t1}\right),{x}_{i,j}^{t}\right)\right\}\) via sliding the time windows forward, which can be used for training the predictor \({f}_{MLP}\). Supposing t = 8 and s = 4, then we would have a sequence of the solution as \({(x}_{i,j}^{1},{x}_{i,j}^{2},\cdots ,{x}_{i,j}^{8}),i=\mathrm{1,2},\cdots ,N,j=\mathrm{1,2},\cdots ,d\), and we can split the sequence as: \(\left\{\left(\left({x}_{i,j}^{1},\cdots,{x}_{i,j}^{4}\right),{x}_{i,j}^{5}\right),\left(\left({x}_{i,j}^{2},\cdots,{x}_{i,j}^{5}\right),{x}_{i,j}^{6}\right)\right.\), \(\left.\left(\left({x}_{i,j}^{3},\cdots,{x}_{i,j}^{6}\right),{x}_{i,j}^{7}\right),\left(\left({x}_{i,j}^{4},\cdots,{x}_{i,j}^{7}\right),{x}_{i,j}^{8}\right)\right\}\).
The pseudocode of training the MLPbased predictor is given as Algorithm 1.
The framework of MOEA/DMLP
The whole framework of our dynamic multiobjective optimization algorithm—MOEA/DMLP consists of three parts: an environmental change detection mechanism, a MLPbased predictor, and a multiobjective optimization algorithm implemented in the static environment. The pseudocode of MOEA/DMLP is shown in Algorithm 2, wherein the static multiobjective optimization algorithm is based on MOEA/DDE. In Line 6, we randomly choose 10% of individuals as sensors to evaluate if the environment changes. If the average objective function of these sensors change over iteration, this is judged as the environmental changing. In Line 10, the MLPbased predictor starts to work from the (s + 2)th time step, which ensures that at least one sample can provide for training the MLP network.
Experimental design
Benchmark and performance metric
We choose the DF Benchmark [47] as the test suite to evaluate our proposed algorithm, wherein the dynamic characteristics simulate the realworld scenarios of DMOPs from various properties. There are fourteen test problems, including nine biobjective and five triobjective problems. The time step t is defined as \(t=1/{n}_{t}*\left\lfloor\tau /{\tau }_{t}\right\rfloor\), wherein \({n}_{t}\),\(\tau \) and \({\tau }_{t}\) represent the change severity, the generation counter, and the change frequency, respectively. In this paper, the severity of change is set as: \({n}_{t}=10, 5\), and the frequency of change is set as \({\tau }_{t}=\mathrm{5,10}\).
To observe the difference between the obtained POF and the real POF in each test suite, the modified Inverted Generational Distance (MIGD) is employed as the performance metric. We first introduce the IGD metric, which is defined as follows:
where \({{\varvec{P}}{\varvec{O}}{\varvec{F}}}^{*}\) is the true POF of a multiobjective optimization problem, \({\varvec{P}}{\varvec{O}}{\varvec{F}}\) is an approximation set of \({{\varvec{P}}{\varvec{O}}{\varvec{F}}}^{\boldsymbol{*}}\) obtained by a multiobjective optimization algorithm, and n is the number of individuals in the \({{\varvec{P}}{\varvec{O}}{\varvec{F}}}^{\boldsymbol{*}}\).
The MIGD is defined as the average of the IGD in a certain number of time steps:
where T is the set of time steps in a single run.
Compared algorithms and parameter settings
In this paper, we compare our proposed MOEA/DMLP with four stateoftheart algorithms in the empirical studies, including PPS [17], MOEA/DKF [16], TrDMOEA [23] and MOEA/DSVR [22]. PPS is the first proposed learningbased method for solving DMOPs, which used a linear regression technology to predict new solutions in dynamic environments. MOEA/DKF is a typical nonlearning method to solve DMOPs, which employed a Kalman Filter (KF) model to predict new solutions. TrDMOEA is a novel transferlearning method which used a domain adaptation approach to build a prediction model. MOEA/DSVR is a recently proposed nonlinear learningbased method wherein the Support Vector Regression was used as the predictor. To compare fairly, the MOEA/D framework are used as the MOEA algorithm in PPS and TrDMOEA. All algorithms use the DE operator and the polynomial mutation operator to generate new solutions.
The population size is set as 100 and 300 for biobjective and triobjective problems, respectively. The parameters in the DE operator are set as CR = 0.5, F = 0.5, and \(\upeta =20,{p}_{m}=1/d\) in the polynomial mutation operator. The number of hidden layers is set to be 10, the number of nodes in the input layer is set to be 4 which represents that the solution in the current environment is correlated with its locations in four previous environments. Since the number of training samples is positively related to the number of environmental changes, online training the MLPbased predictor would be timeconsuming beginning from some time steps due to with many training samples. Here we set a hyper parameter to control the number of training samples beginning from the 45th environmental change, which ensures that no more than 40 samples are used to train the predictor.
Results and discussion

(1)
Solution Quality: Tables 1 and 2 present the experimental results of MOEA/DMLP and the comparing algorithms in terms of MIGD values on DF1DF14. A smaller value of n_{t} signifies that the benchmark changes in a larger severity, which causes the algorithms to track the changing POS and/or POF much harder. The MIGD values obtained on benchmarks with n_{t} = 5 would be worse. A smaller value of τ_{t} implies that the benchmark changes more frequently, the algorithms are given less evaluations to approximate new POS. Therefore, the MIGD values obtained on the problems with τ_{t} = 10 would be better. It is clear that MOEA/DMLP achieves superior performance in terms of the MIGD values against the other methods on most of benchmarks under different dynamic configurations. The POS of DF2 and DF3 change over time with simple dynamic characteristics, and the POF remains stationary over time, but the switch of the positionrelated variable is challenging that causes the correlations of positions over time are not clear. TrDMOEA performs best on DF2 and DF3, instead of assuming any correlations of solutions between the consecutive time steps, it predicts a distribution of solutions. Therefore, TrDMOEA is more suitable for solving problems with a stationary POF or similar POF in sequential environments. Since of specific dynamic characteristics of DF6 and DF7, it is hard to approximate POS within the given evaluation budget. Thus the historical solutions can not indicate the hidden correlation between solutions over time steps. All algorithms perform badly comparing with their performance on the other benchmarks. For the triobjective optimization problems DF10DF14, our algorithm MOEA/DMLP performs competitively against the other algorithms. It is worth noting that MLP and SVR are both typical machine learning methods, and the MLPbased predictor performs better than the SVRbased predictor on most of these benchmarks. The better generalization capability of MLP may contribute to the superiority of our predictor.

(2)
Tracking Curve: To access the tracking performance of these algorithms on DMOPs, we plot the tracking of the IGD values over environmental changes by five algorithms on DF1, DF2, DF5, DF9, DF11 and DF12 with τ_{t} = 10, n_{t} = 5, showing the average values over 20 runs. Observed from Fig. 3a, MOEA/DMLP and MOEA/DSVR performs more stably on DF1 after 30 environmental changes, and have similar tracking performance. TrDMOEA performs best on DF2 in terms of IGD values over time steps observed from Fig. 3 (b). MOEA/DMLP performs similar to MOEA/DSVR in terms of tracking curves, but has better convergence accuracy on most of these benchmarks observed from Fig. 3c–f.

(3)
Converge Curve: To investigate the convergence performance of these algorithms, we plot the IGD values over generations obtained by five algorithms on DF1, DF2, DF5, DF9, DF11 and DF12 with τ_{t} = 10, n_{t} = 5, showing the average values over 20 runs. To show the tracking process of these algorithms clearly, only the IGD values over the first 200 generations are presented, in which 20 environmental changes happen. Observing from Fig. 4a, MOEA/DMLP, PPS and MOEA/DSVR have much better final IGD values than TrDMOEA and MOEA/DKF in most of environments. Starting from the sixth environment, the MLPbased predictor can provide better initial population. Since of good initial population, MOEA/DMLP can better approximate POF in the objective space and obtain a much lower IGD value in the end of each environment. In the later time steps, MOEA/DMLP has the best IGD values in the initial stage and the end stage for the most of time steps. In Fig. 4b, TrDMOEA performs most stably, and MOEA/DMLP has stable tracking ability beginning from the 14th environmental change. Viewing from Fig. 4c–f, our algorithm MOEA/DMLP also performs competitively comparing with MOEA/DSVR and PPS in terms of convergence curves.

(4)
Influence of the number of nodes in the input layer and hidden layer. In our MLPbased predictor, the number of nodes in the input layer and hidden layer is set to 4 and 10, respectively. To investigate how MOEA/DMLP is sensitive to these two parameters, we study four different numbers of nodes in the input layer and hidden layer, respectively. Table 3 presents the results of varying the number of nodes in the input layer of MOEA/DMLP on some benchmarks, which illustrates that too small or too large number of nodes cannot bring good benefits for our algorithm. A larger number of nodes in the input layer represents the target solution is correlated with many preceding solutions, which does not improve the performance of the MLPbased predictor. Table 4 presents the results of varying the number of nodes in the hidden layer of MOEA/DMLP on some benchmarks. The performance of MOEA/DMLP should be improved by with a larger number of nodes in the hidden layer theoretically, which however relies on more iterations for training the network and more training data. Thus, the limited training data and iterations for training on these benchmarks cannot always make our algorithm with larger number of nodes acquire better results.

(5)
Discussion on Running Time: The learningbased methods require online training on historical data, which costs extra running time, thus it is worth to discuss the running time on training the predictors. Let’s take running on DF1 for example, training an AR model in PPS needs 8 s average over all environmental changes. While the average training time on a SVR model costs 3 s wherein 40 training samples are retained. As a contrast, the average training time on our MLPbased predictor costs 10 s. However, the domain adaptation in TrDMOEA needs much more time reaching to 210 s average over all environmental changes. Although training MLPbased predictor costs more time than training AR and SVR model, the MLPbased predictor provides better initial population for the MOEA/D algorithm in dynamic environments.
Conclusion
In this paper, we present a deep multilayer perceptron based predictor associated with the framework of MOEA/D to solve DMOPs. In several initial environments, MOEA/D collects the approximated solutions as the training data. In the beginning of a new environment, the training data is used to train a deep MLPbased predictor to fit the hidden correlations within the training data, and predict new solutions as the initial population for MOEA/D in this new environment. The motivation of this predictor is to assume there are some hidden correlations between POS in consecutive environments, and we attempt to use a function to describe this correlations. Theoretically, the MLP can fit any function among the given data, which therefore can be used to describe the hidden correlations between the approximated POS by MOEA/D from several environments. Experimental results demonstrate that our proposed algorithm is effective in tracking varying solutions over time and shows great superiority comparing with stateoftheart methods. There are still several possible directions for future work. The performance of MLPbased predictor greatly depends on the quality of the historical data, which may result in bad performance if the environment change rapidly or severely.
References
Jin Y, Branke J (2005) Evolutionary optimization in uncertain environmentsa survey. IEEE Trans Evol Comput 9(3):303–317
R. Azzouz, S. Bechikh, and L. Ben Said, “Dynamic multiobjective optimization using evolutionary algorithms: a survey,” in Recent Adavances in Evolutionary Multiobjective Opimitzation.Adaptation,Learning,and Optimization, vol. 20, S. Bechikh, R. Datta, and A. Gupta, Eds. Springer International Publishing Switzerland 2017, 2017, pp. 31–70.
R. Azzouz, S. Bechikh, and L. Ben Said, “Multiobjective optimization with dynamic constraints and objectives : new challenges for evolutionary algorithms,” in Proceedings of ACM GECCO conference, 2015, pp. 615–622.
Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGAII. IEEE Trans Evol Comput 6(2):182–197
A. Zhou, Y. Jin, Q. Zhang, B. Sendhoff, and E. Tsang, “Predictionbased population reinitialization for evolutionary dynamic multiobjective optimization,” in Proceedings of EMO,LNCS4403, 2007, pp. 832–846.
Hatzakis I, Wallace D (2006) Dynamic multiobjective optimization evolutionary algorithms: a ForwardLooking approach. Proceedings of GECCO 4:1201–1208
Nguyen S, Zhang M, Johnston M, Tan KC (2014) Automatic design of scheduling policies for dynamic multiobjective job shop scheduling via cooperative coevolution genetic programming. IEEE Trans Evol Comput 18(2):193–208
Zhu Z, Gao X, Cao L, Cai Y, Pan D (2016) Research on the shift strategy of HMCVT based on the physical parameters and shift time. Appl Math Model 40:6889–6907
Zhang Z (2008) Multiobjective optimization immune algorithm in dynamic environments and its application to greenhouse control. Appl Soft Comput 8(2):959–971
Jiang S, Yang S (2017) Evolutionary Dynamic Multiobjective Optimization: Benchmarks and Algorithm Comparisons. IEEE Trans Cybern 47(99):198–211
Jiang S, Yang S (2017) A steadystate and generational evolutionary algorithm for dynamic multiobjective optimization. IEEE Trans Evol Comput 21(1):65–82
Gee SB, Tan KC, Alippi C (2017) Solving Multiobjective Optimization Problems in Unknown Dynamic Environments: An Inverse Modeling Approach. IEEE Trans Cybern 47(12):4223–4234
Gee SB, Tan KC, Abbass HA (2017) A benchmark test suite for dynamic evolutionary multiobjective optimization. IEEE Trans Cybern 47(2):461–472
N. Saxena, K. K. Mishra, and A. Tripathi, “DWTSVDBased Color Image Watermarking Using DynamicPSO,” in Advances in Computer and Computational Sciences, Advances in Intelligent Systems and Computing, 2018, pp. 343–351.
Ma X, Yang J, Sun H, Hu Z, Wei L (2021) Multiregional coevolutionary algorithm for dynamic multiobjective optimization. Inf Sci (Ny) 545:1–24
Muruganantham A, Tan KC, Vadakkepat P (2016) Evolutionary dynamic multiobjective optimization via Kalman filter prediction. IEEE Trans Cybern 46(12):2862–2873
Zhou A, Jin Y, Zhang Q (2014) A population prediction strategy for evolutionary dynamic multiobjective optimization. IEEE Trans Cybern 44(1):40–53
Zou J, Li Q, Yang S, Bai H, Zheng J (2017) A prediction strategy based on center points and knee points for evolutionary dynamic multiobjective optimization. Appl Soft Comput 61:806–818
L. Cao, L. Xu, E. D. Goodman, S. Zhu, and H. Li, “A differential prediction model for evolutionary dynamic multiobjective optimization,” in Proceedings of GECCO, 2018, pp. 601–608.
Wu Y, Jin Y, Liu X (2015) A directed search strategy for evolutionary dynamic multiobjective optimization. Soft Comput 19(11):3221–3235
M. Jiang, W. Hu, L. Qiu, M. Shi, and K. C. Tan, “Solving Dynamic Multiobjective Optimization Problems via Support Vector Machine,” in 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), 2018, vol. 1, pp. 819–824.
L. Cao, L. Xu, E. D. Goodman, C. Bao, and S. Zhu, “Evolutionary Dynamic Multiobjective Optimization Assisted by a Support Vector Regression Predictor,” IEEE Trans. Evol. Comput., vol. 24, no. 2, 2020.
Jiang M, Huang Z, Qiu L, Huang W, Yen GG (2018) Transfer learning based dynamic multiobjective optimization algorithms. IEEE Trans Evol Comput 22(4):501–514
Jiang M, Qiu L, Huang Z, Yen GG (2018) Dynamic multiobjective estimation of distribution algorithm based on domain adaptation and nonparametric estimation. Inf Sci (Ny) 435:203–223
Jiang M, Wang Z, Hong H, Yen GG (2021) Knee PointBased Imbalanced Transfer Learning for Dynamic Multiobjective Optimization. IEEE Trans Evol Comput 25(1):117–129
Ramchoun H, Amine M, Idrissi J, Ghanou Y, Ettaouil M (2016) Multilayer Perceptron: Architecture Optimization and Training. Int J Interact Multimed Artif Intell 4(1):26
Zhang Q, Li H (2007) MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evol Comput 11(6):712–731
Li H, Zhang Q (2009) Multiobjective optimization problems with complicated pareto sets, MOEA/D and NSGAII. IEEE Trans Evol Comput 13(2):284–302
Ruan G, Yu G, Zheng J, Zou J, Yang S (2017) The effect of diversity maintenance on prediction in dynamic multiobjective optimization. Appl Soft Comput 58:631–647
Cao L, Xu L, Goodman ED, Li H (2019) Decompositionbased evolutionary dynamic multiobjective optimization using a difference model. Appl Soft Comput 76:473–490
R. Rambabu, P. Vadakkepat, K. C. Tan, and M. Jiang, “A mixtureofexperts prediction framework for evolutionary dynamic multiobjective optimization. IEEE Trans Cybern pp. 1–14, 2019.
M. Rong, D. Gong, W. Pedrycz, and L. Wang, “A MultiModel Prediction Method for Dynamic MultiObjective Evolutionary Optimization,” IEEE Trans. Evol. Comput., pp. 1–15, 2019.
M. Rong, D. Gong, Y. Zhang, Y. Jin, and W. Pedrycz, “Multidirectional Prediction Approach for Dynamic Multiobjective Optimization Problems,” IEEE Trans. Cybern., pp. 1–13, 2018.
C. Wang, G. G. Yen, and M. Jiang, “A grey predictionbased evolutionary algorithm for dynamic multiobjective optimization,” Swarm Evol. Comput., vol. 56, no. April, p. 100695, 2020.
Hu Y, Zheng J, Zou J, Yang S, Ou J, Wang R (2020) A dynamic multiobjective evolutionary algorithm based on intensity of environmental change. Inf Sci (Ny) 523:49–62
Wang F, Liao F, Li Y, Wang H (2021) A new prediction strategy for dynamic multiobjective optimization using Gaussian Mixture Model. Inf Sci (Ny) 580:331–351
Z. Liang, T. Wu, X. Ma, Z. Zhu, and S. Yang, “A Dynamic Multiobjective Evolutionary Algorithm Based on Decision Variable Classification,” IEEE Trans. Cybern., pp. 1–14, 2020.
M. Jiang, Z. Wang, S. Guo, X. Gao, and K. C. Tan, “IndividualBased Transfer Learning for Dynamic Multiobjective Optimization,” IEEE Trans. Cybern., pp. 1–14, 2020.
H. Zhang and G.G. Wang, “Improved NSGAIII using transfer learning and centroid distance for dynamic multiobjective optimization,” Complex Intell. Syst., 2021.
Zou F, Yen GG, Tang L, Wang C (2021) A reinforcement learning approach for dynamic multiobjective optimization. Inf Sci (Ny) 546:815–834
Wang F, Li Y, Liao F, Yan H (2020) An ensemble learning based prediction strategy for dynamic multiobjective optimization. Appl Soft Comput J 96:106592
L. Feng, W. Zhou, W. Liu, Y.S. Ong, and K. C. Tan, “Solving Dynamic Multiobjective Problem via Autoencoding Evolutionary Search,” IEEE Trans. Cybern., pp. 1–14, 2020.
A. A. Heidari, H. Faris, S. Mirjalili, I. Aljarah, and M. Mafarja, Ant lion optimizer: Theory, literature review, and application in multilayer perceptron neural networks, vol. 811. Springer International Publishing, 2020.
Wan S, Liang Y, Zhang Y, Guizani M (2018) Deep MultiLayer perceptron classifier for behavior analysis to estimate Parkinson’s disease severity using smartphones. IEEE Access 6:36825–36833
Samadianfard S et al (2020) Wind speed prediction using a hybrid model of the multilayer perceptron and whale optimization algorithm. Energy Rep 6:1147–1159
Zhang Y, Fu F, Chen X, Xie G, Wang L (2015) Cooperation in groupstructured populations with two layers of interactions. Sci Rep 5:17446
S. Jiang, S. Yang, X. Yao, K. C. Tan, and M. Kaiser, “Benchmark Problems for CEC2018 Competition on Dynamic Multiobjective Optimisation,” 2018.
Acknowledgements
This work was supported in part by the Open Fund of State Key Laboratory of Power System of Tractor under Grant SKT2022011, in part by the National Natural Science Foundation of China under Grant 51805222 and 51875255 and U20A20331, in part by the Open Fund of State Key Laboratory of Automotive Simulation and Control under Grant 20201201, and in part by the Project of Jiangsu Provincial Six Talent Peaks under Grant 2018TDGDZB022.
Author information
Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhu, Z., Yang, Y., Wang, D. et al. Deep multilayer perceptronbased evolutionary algorithm for dynamic multiobjective optimization. Complex Intell. Syst. (2022). https://doi.org/10.1007/s40747022007452
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40747022007452
Keywords
 Dynamic multiobjective optimization
 Evolutionary algorithm
 Prediction
 Deep multilayer perceptron