A federated data-driven evolutionary algorithm for expensive multi-/many-objective optimization

Data-driven optimization has found many successful applications in the real world and received increased attention in the field of evolutionary optimization. Most existing algorithms assume that the data used for optimization are always available on a central server for construction of surrogates. This assumption, however, may fail to hold when the data must be collected in a distributed way and are subject to privacy restrictions. This paper aims to propose a federated data-driven evolutionary multi-/many-objective optimization algorithm. To this end, we leverage federated learning for surrogate construction so that multiple clients collaboratively train a radial-basis-function-network as the global surrogate. Then a new federated acquisition function is proposed for the central server to approximate the objective values using the global surrogate and estimate the uncertainty level of the approximated objective values based on the local models. The performance of the proposed algorithm is verified on a series of multi-/many-objective benchmark problems by comparing it with two state-of-the-art surrogate-assisted multi-objective evolutionary algorithms.


Introduction
Many optimization problems have several conflicting objectives that need to be optimized concurrently, which are known as multi-objective optimization [11], and many-objective optimization when the number of objectives is larger than three [28].Over the past decades, a pile of successful multi-objective evolutionary algorithms (MOEAs) have been developed, which can largely be categorized into decomposition-based MOEAs, such as the multi-objective evolutionary algorithm based on decomposition (MOEA/D) [50] and reference vector guided evolutionary algorithm (RVEA) [6]; Pareto-dominance based, e.g., the elitist non-dominated sorting genetic algorithm (NSGA-II) [12]; and performance indicator-based [14].Note that many MOEAs combine decomposition or Pareto dominance with other criteria to address various challenges in multi-objective optimization [52].
It is well recognized, however, that most MOEAs require a large number of function evaluations before a set of diverse and well-converged non-dominated solutions can be found, which makes it hard for them to be directly applied to solve a class of data-driven optimization problems [24], whose objectives can be evaluated by means of conducting time-consuming computer simulations or costly physical experiments.Examples of real-world data-driven multi-objective optimization problems include blast furnace optimization [7], air intake ventilation system optimization [10], airfoil design optimization [29], bridge design optimization [34], design optimization for a stiffened cylindrical shell with variable ribs [54], optimization of operation of crude oil distillation units [19], resource allocation of trauma systems [44], and also neural architecture search [39,55], among many others.
Data-driven surrogate-assisted evolutionary algorithms were first developed for single-objective optimization [23].In recent years, more attention has been paid to data-driven surrogate-assisted multi-objective optimization [2,9].Similar to MOEAs, surrogate-assisted MOEAs can also be divided into four groups.The first group is based on decomposition, where surrogates are built to approximate the sub-problems using random weights, known as ParEGO [27].Alternatively, surrogates can also be constructed for each individual objective function to assist a decomposition based MOEA, such as the Gaussian process assisted MOEA/D, called MOEA/D-EGO [51], K-RVEA [8], a Kriging-assisted RVEA, and surrogate-ensemble assisted MOEA, called SAEMO [53].The second group employs the non-dominance relation for environmental selection, and surrogates are adopted for approximating the objective functions [43,18], or in local search [31].In [32,35], a surrogate is built for estimating the dominance relationship, and consequently only a single surrogate is required.The third category relies on a performance indicator-based MOEA, e.g., a Gaussian process assisted hypervolume based MOEA [15], and an enhanced inverted generational distance based MOEA assisted by a dropout deep neural network (EDN-ARMOEA) [19].Finally, surrogates have also been applied to assist other MOEAs, such as the surrogate-assisted two-archive evolutionary algorithm [37].A good overview of data-driven evolutionary optimization can be found in a recently published book [25].
Despite its impressive success, data-driven evolutionary optimization is faced with several challenges.One is the lack of data, which is very expensive to collect.To address this issue, advanced learning techniques such as semi-supervised learning [38,17,21], knowledge transfer between different surrogates using transfer optimization [49,20], and between different objectives using transfer learning [45,46] have been adopted in surrogate-assisted optimization.Another solution is to reduce the computational cost of data collection by parallelizing expensive computer simulations [1,5] in data-driven surrogate-assisted evolutionary optimization.
Little attention has been paid to data privacy concerns in data-driven optimization.One exception is our previous work reported in [48], which proposes a federated data-driven evolutionary algorithm (FDD-EA) for single-objective optimization based on federated learning [33,56].As its name suggests, federated learning aims to collaboratively train a global model using data distributed on multiple local devices without transmitting the local data to a central server.Instead, each client train a local model based on the local data and then the parameters (or the gradients) of the local models are transmitted to the server to generate the global model.Similarly, in FDD-EA, the central server averages the local models uploaded by the clients with a sorted averaging algorithm [48].Then, the global and local models are made use of for approximating the objective function and estimating the uncertainty of the approximated fitness, based on which an infill criterion is designed for surrogate management in the federated optimization environment.By extending FDD-EA, this work proposes a federated data-driven evolutionary algorithm for multi-objective optimization.The main contributions of this paper are summarized as follows.
-A federated data-driven multi-/many-objective evolutionary optimization algorithm (FDD-MOEA) is presented, in which multiple clients collaboratively train a global surrogate without uploading local data.To the best of our knowledge, this is the first work that enables federated data-driven multi/many-objective optimization that does not require to store the local data on a central server, thereby preserving the privacy of the data.-An ensemble-based acquisition function is adopted for FDD-MOEA, in which the global surrogate is used to provide the approximated fitness and the local surrogates are used to provide an estimate of the uncertainty of the approximated fitness.
The remainder of this paper is arranged as follows.Section 2 presents the concepts of multi-objective optimization, federated learning and acquisition function.The main workflow and mechanisms of the proposed method, FDD-MOEA, are illustrated in Section 3.Then, various empirical studies on test suites are given in Section 4. Finally, Section 5 summarizes the contributions and future directions of this paper.

Background and Motivations
In this part, we briefly illustrate the concept of multi-objective optimization, RBFN surrogate, federated learning and acquisition function.

Multi-objective optimization
Denote the minimization of vector of objectives by where x ∈ R d is the decision vector with dimension d, M is the number of objectives and M ≥ 2, and the profile formed by the projections of the aforementioned Pareto optimal solutions in the objective space is called Pareto front (PF).Note that this work focuses on the optimization of unconstrained MOPs.

Radial basis function networks
Radial basis function networks (RBFNs) [13] are widely adopted surrogate models due to their high computational efficiency and robustness on high dimensional problems.The first hidden layer connected to the input of an RBFN is composed by a series of centers {c 1 , ..., c i , ..., c q }, where q is the number of centers, and c i ∈ R d is the i th center.This layer calculates the distance between a sample point , where φ(•) represents a radial basis function, and in this work we use Gaussian functions with a spread δ i , Suppose X ∈ R n k ×d and Y ∈ R n k ×M represent a dataset with n k (indexing by j) sample pairs (x j ∈ R d , y j ∈ R M ), then the corresponding output of the RBFN can be calculated by which can further be simplified to be ŷj = Φ j w + b, for j = 1, 2, ..., n k where denotes the distance between sample x j and all centers, and w = [w 1 , .., w M ] ∈ R q×M , in which w M = [w 1,M , w 2,M , ..., w q,M ] T are the weights of the connecting M th attribute of y j and all centers, and b = [b 1 , ..., b M ] ∈ R 1×M is the bias, δ = {δ 1 , δ 2 , ..., δ q } ∈ R q stands for the spread vector of the RBFN.Hence, we can use [w, δ, c, b] to represent the RBFN.
In terms of the training process, the centers of an RBFN is usually determined by random selection or K-means clustering.The least square method and backpropagation are two most popular methods [13] to train the weights w and biases b.The least square method optimizes the weights and the biases through matrix inversion (linear algebra), whereas backpropagation uses strategies such as the stochastic gradient descent (SGD) method [36] to iteratively optimize the parameters.
Compared with GPs, RBFNs are efficient for training and it is less sensitive to the increase of training data size.However, an RBFN cannot directly provide the uncertainty of its predictions like GPs, making it unsuitable for applying an infill criterion for surrogate model management.

Federated learning
Federated learning [33] originates from the attempt to address the privacy concerns in distributed learning [30], which has been widely studied and applied in real-world applications due to its capability of privacy protection and parallel computing [47,56].Generally, the global optimization objective of a federated learning system can be written by where N is the total number of clients, λ is a coefficient that reflects how many clients will participate in model update in each communication round, and p k is the weight that controls the importance of the participating client k, which is usually determined by the size of local dataset D k , and F k denotes the local objective, which is where L is the loss function.
As Mcmahan [33] suggested, the global model W s can be conducted by averaging a set of local models W k (k = {1, 2, ..., λN }) rather than collecting distributed local data, which can be denoted by To summarize, federated learning is a distributed learning framework, in which a local model is trained on each client.The trained local models are then uploaded onto the server and aggregated into a global model.The global model is then downloaded to all participating clients to be further trained.Ideally, the global model will converge to the one that is trained on all local data that are centrally stored.
It should be emphasized that the main motivation of the present work is to achieve data-driven optimization of complex systems, in which the data is collected and distributed on multiple clients.Due to privacy concerns and other considerations, the data is not allowed to transmitted to a central server.
To the best of our knowledge, all existing work on data-driven evolutionary optimization assumes that the sampled data are available for training a global surrogate model using centralized learning, with one exception reported very recently in [48], which proposed, for the first time, an algorithm for federated data-driven single-objective optimization.Actually, this work is a natural extension of [48], aiming to propose a federated data-driven evolutionary multi-objective optimization algorithm, FDD-MOEA for short.

Acquisition function
Acquisition function (AF), also called infill criterion or model management, is designed to decide where to sample new data (new candidate solutions) in the decision space, i.e., to determine which candidate solutions x should be selected for calculating their objective values f (x), usually by means of time-consuming numerical simulations or physical experiments.Acquisition functions were originally developed in Bayesian optimization when the Gaussian process is used as the surrogate, although it can be easily extended to any other types of surrogates with the capability of providing uncertainty information, such as ensemble surrogates [18], or dropout neural networks [19].
An infill criterion must be able to balance the exploitation and exploration [26], and several successful infill criteria have been widely adopted by surrogate-assisted MOEAs, such as Expectation Improvement (EI) [26] and Lower Confidence Bound (LCB) [41].In this work, we choose LCB to minimize Eq. ( 9), which is where α is a hyperparameter which is set to 2 as suggested in [16] in the experimental studies.

Federated Data-driven Multi-objective Optimization
In this section, we will briefly introduce the federated construction of the global model, the workflow of the proposed FDD-MOEA, and the used federated infill criterion.

Federated surrogate construction
Clients: represents the surrogate on the k th client, and let { ŷ1 , ŷ2 , ..., ŷB } represent the predicted outputs with a batch size of B (1 then the loss function (L in Eq. ( 7)) of each local RBFN used in this work can be written as and the centers and spreads of the RBFN are determined by K-means maximum distance between centers, then the gradients of weights w k and b k are calculated by By replacing ŷi with Eq. ( 10), we have Server: After the local model on the k-th client (W k ) is obtained and uploaded to the server, the sorted averaging algorithm, as described in Algorithm 1 and first reported in [48], will be adopted to aggregate all received local models to create the global model W s .Then, the server will use W s to assist an MOEA.Finally, W s and the found potential solution x to the chosen clients, the k th client will evaluate the x by the expensive objectives and update the local dataset to further update local model W k , which can be illustrated by Fig. 1.The overall framework will be described in Section 3.2.

Overall framework
As shown in Fig. 2, the proposed FDD-MOEA differs from those of ensemblebased methods in that it utilizes sorted model averaging to obtain a global model W s for fitness approximation instead of averaging predictions of ensemble.In addition, the data partitions for each local models are determined by the operating conditions of the clients and cannot be actively manipulated as done conventional ensemble construction when the data is centrally stored.It also be stress that in principal, any MOEAs can be applied to the proposed framework.In our experimental studies, we choose NSGA-II and RVEA to examine the performance of the proposed framework for MOPs and MaOPs, respectively.
Algorithm 2 summarizes the overall workflow of the proposed FDD-MOEA.Before the optimization starts, Latin hypercube sampling (LHS) is adopted to generate 11d − 1 points in the decision space, where d is the dimension of the decision space.And these initial points are evaluated by the expensive real objective functions (data collection), the collected data pairs are used as the initial training dataset of for the local models.Then, the k th client will train the local RBFN W k with its own dataset and upload the trained W k to the server.The server aggregates all uploaded local models by means of sorted averaging to obtain the global model W s .After that, the acquisition function (AF) will be formed using W s and all local models (refer to Section 3.3), in which the averaged W s will replace the expensive objective functions to Fig. 2 The overall framework of the proposed FDD-MOEA.
predict the objective values, and the local models will collaboratively estimate the uncertainty of the predictions.
Then, an MOEA is chosen to optimize the acquisition functions, and m new solutions (specified by the user) will be selected from the results of MOEA through clustering.Note, however, that duplicate solutions will be removed before clustering.
1) Removal of redundant solutions.It is very common when the optimization completes, the final population of the MOEA contains redundant solutions or extremely similar solutions.In this work, we calculate the Euclidean distances between solutions of the current population in the decision space and only one will be kept if the distance between two solutions is less than a threshold (10 −6 in this work).Note that the number of solutions after removing the redundant solutions may be smaller than m, in particular for high-dimension problems.In this case, we will save the existing solutions and restart the MOEA until the stopping criterion is satisfied.2) Clustering.After removing the redundant solutions, the remaining ones will be clustered using the K-means algorithm with a center size of m.
In federated optimization, there are many challenges that may lead to difficulties in collecting well distributed data, making it hard to achieve a highquality global model.In this work, we consider two main challenges, namely communication failures and client dropout.We define a probability of communication failure p f to denote whether the communication between a certain client and the server is successful in one particular iteration of surrogate management.If the communication is successful, then all m candidate solutions selected by the server will be passed to the participating clients, evaluated, and added to the local training dataset for the next round training.Otherwise, the m new solutions will not be evaluated by that particular client.Consequently, the larger the communication failure probability, the stronger the data will be non-independent and identically distributed (non-iid).
Client dropout is also caused by communication problems.Like in federated learning, we use a participation ratio λ to define the percentage of clients that participate in each round of model update and surrogate management.The smaller participation ratio, the smaller the number of local models, which may result in a poorer global model.
To be consistent with the settings in [18], we also limit the maximum number of training data to be l = 11d − 1 + 25.A fast non-dominated sorting is applied to select l data for model training.

Modified LCB
As we stated in Section 2.4, the acquisition function prefers solutions with high uncertainty and better performance.To achieve this, we utilize the global surrogate model together with the local models to provide predictions of candidate solutions and estimate the uncertainty of the predictions.
First, once the server model W s is aggregated, the evaluated objective values f (x) in Eq. ( 9) is calculated by Similar to the ensemble-based strategy, the variance ŝ2 (x) of the predictions can be estimated by Consequently, we can easily calculate the LCB values for optimization using the MOEA.

Numerical Studies
In the following, the benchmark problems and parameter settings used in the empirical studies are introduced and then the comparative results are presented.

Benchmark problems
As listed in Table 1, we adopt the DTLZ test suite for comparative studies, and 49 test instances in total with different number of decision variables and objectives are used.According to [22], an MOEA may get trapped in local Pareto front of DTLZ1 and DTLZ3, which will lead to poor performance.And the PF of DTLZ2 are located in an octant of the unit sphere, which is often used to test the performance of an MOEA with a large number of objectives.It is hard for an MOEA to find evenly distributed solutions in the objective space for DTLZ4 due to the biased projection from decision space to the objective space [18], and the mapping parameter α of DTLZ4 is set to 100 in this work.DTLZ5 and DTLZ6 have degenerate Pareto optimal fronts, and each front is supposed to be an arc in the objective space.The Pareto front of DTLZ7 is disconnected.

Parameter settings
Recall that the number of objectives and the dimension of the decision variable are denoted by M and d, respectively.The basic parameter settings are as follows.
1.The number of independent runs is set to 20 for each test instance, and each time we sample 11d − 1 points in the decision space using LHS. 2. NSGA-II is chosen as the MOEA for MOPs, the population size and the number of generations are both set to 50.The crossover distribution index and its probability are set to 20, 1.0.And the corresponding mutation distribution index and probability are 20 and 1/d. 3. RVEA is adopted as the MOEA for MaOPs, and the number of reference vectors are set to 105, 126, 275, 420 for 3, 5, 10, 20 objectives, respectively.4. As suggested in [18], the number of RBFN centers is limited to √ M + d + 3, the optimization process starts at 11d − 1 fitness evaluations (FEs) and ends at 11d−1+120 FEs, and five new FEs are performed (five new training samples are collected) in each round, which means that the number of iteration R is 24 in a single run.The maximum size of training data l is set to 11d − 1 + 25. 5.The hyperparameters of federated learning are set as follows: the local epoch number E = 20, the learning rate η = 0.06, the number of clients N = 10, and the participation ratio λ = 0.9.As previously discussed, for each individual client, the solutions to be sampled at each iteration as identified by the surrogate management strategy by the server may not be implemented on each client with a communication failure probability p f = 3%.To make a fair comparison with ensemble based algorithms, the initial solutions are the same on all clients.6. Inverted generational distance (IGD) [4] is adopted as the performance indicator, which calculates the Euclidean distance between the non-dominated solutions obtained by an algorithm and a set of reference points sampled from the theoretical Pareto front.The result of 20 independent runs are analyzed by the Wilcoxon rank sum at a significance level of 0.05, where symbol '+' means that FDD-MOEA outperforms the compared method, '=' indicates the two algorithms perform equally well, and '−' means that the compared method outperforms FDD-MOEA.

Results and discussions
We implement the proposed FDD-MOEA with Python 3.7 [42], and the RBFN is constructed from scratch using the stochastic gradient descent method.The code of NSGA-II is taken from Pymoo [3], and the K-RVEA are implemented by PlatEMO v2.8.0 [40], and the performance indicators are also calculated on PlatEMO.All experiments are conducted on a computer with Intel(R) Core(TM) i7-8700 CPU@3.20GHz3.19 GHz, Windows 10, version 1909.

Performance comparison on three-objective test instances
The first set of experiments compares FDD-MOEA with three centralized data-driven MOEAs, namely HeE-MOEA [18], GP-MOEA [18,19] and K-RVEA [8] on three-objective DTLZ functions with a dimension of the decision spacce d = 10, 20, 40, 80. Specially, HeE-MOEA utilizes a heterogeneous ensemble to assist the optimization, while GP-MOEA is a variant of HeE-MOEA that adopts a GP as the surrogate for approximating each objective function.Note that HeE-MOEA, GP-MOEA and FDD-MOEA use NSGA-II as the baseline optimizer, while K-RVEA is based on RVEA.The average IGD results on all these test instances are summarized in Table 2.
From the results in Table 2, we can observe that FDD-MOEA obtains the best IGD value on 16 out of 28 test instances, compared with HeE-MOEA, GP-MOEA and K-RVEA.These results confirm that the proposed FDD-MOEA works effectively for solving MOPs in a federated system.We also note that K-RVEA performs better than HeE-MOEA and GP-MOEA.It should be stressed that that the purpose of proposing FDD-MOEA is not for developing a new MOEA that performs better than the state-of-the-art data-driven optimization algorithms.Rather, it aims to propose an algorithm that can still work effectively when the data are distributed on different clients and are not allowed to centrally stored.
The convergence profiles of the compared algorithms on DTLZ1 -DTLZ7 when d = 10, 80 over the iterations are plotted in Figs.3-9.From these results, we find that the convergence properties of different algorithms strongly differ on different test instances.One general conclusion is that the performance advantage of FDD-MOEA over the compared algorithms will become stronger as the search dimension increases.This might be attributed to the use of the RBFNs in FDD-MOEA, which has been demonstrated powerful for highdimensional systems in previous research.

Performance comparison on MaOPs
To further examine the effectiveness of the proposed FDD-MOEA on distributed optimization of many-objective problems, we conduct experiments by setting the number of objectives M = 5, 10, 20 for a fixed dimension d = 20.In this case, all algorithms under comparison use RVEA as the baseline optimizer.Table 3 presents the statistical results in terms of the IGD values.Fig. 9 The IGD convergence profiles on DTLZ7.
Overall, FDD-MOEA performs slightly better than HeE-MOEA and K-RVEA, and much better than GP-MOEA on the MaOP test instances.Compared with HeE-MOEA, FDD-MOEA performs better on 10 out of 21 test instances, equally well and worse on four and seven instances, respectively.Similar to the results presented above, the performance of FDD-MOEA is consistently competitive on all DTLZ1-DTLZ5 instances.FDD-MOEA performs particularly well on DTLZ6 when RVEA is adopted as the optimizer, in contrast to its performance when NSGA-II is used.On DTLZ7, FDD-MOEA is outperformed by HeE-MOEA.In addition, FDD-MOEA outperforms GP-MOEA on 15 out of 21 test instances, achieves similar results on three instances, and is underperformed on four instances.
In general, K-RVEA achieves the second best results on many-objective problems, and K-RVEA outperforms FDD-MOEA on the DTLZ2 and DTLZ7 test instances.Interestingly, when using RVEA as the optimizer, the performance of FDD-MOEA on DTLZ6 problems has a significant improvement and and it outperforms K-RVEA in all cases, which means that using RVEA as the optimizer makes FDD-MOEA more competitive when compared with K-RVEA.

Influence of λ
The participation ratio, λ, is an important parameter in federated learning systems.In federated data-driven optimization, a low participation rate will lead to a decrease in the amount of training data on each client.To examine the influence of the participation ratio λ on the performance of FDD-MOEA, empirical experiments are conducted by varying the values of λ, while the number of decision variables is fixed to d = 30 and RVEA is adopted as the optimizer.
As listed in Table 4, the results indicate that FDD-MOEA is relatively insensitive to the participation ratio and in general, a larger participation ratio contributes to better performance, which is expected.Furthermore, we also investigate the influence of λ on the performance of FDD-MOEA DTLZ1-7 problems in terms of IGD for M = 20, d = 30.The performance changes of FDD-MOEA over different λ values are presented in Fig. 10.The experimental results show that higher participation ratios will lead to an improvement in the performance of FDD-MOEA, mainly because the total amount of selected solutions is limited, and the amount of local training data on each client may significantly decrease with a lower participation ratio λ.

Influence of p f
The aim of federated data-driven EAs is to solve of distributed optimization problems without collecting private data stored on local clients [48] and hence, communications between clients and the server when exchanging surrogate information are unavoidable.However, there are many potential reasons that may cause communication failures or packet losses, such as network congestion or when communication network is not available.As mentioned in Section 3.2, we assume that each client has a failure probability of p f when communicating with the server, which means that the m solutions obtained in this iteration cannot be passed to the client.In this part, we study the impact of p f on the performance of FDD-MOEA, and the experimental results with p f = 1%, 3%, 5% and 10% are summarized in Table 5, and the converged IGD values of the compared algorithms are plotted in Fig. 11 when M = 20.Specifically, the dimension of the decision space d and the participation ratio are fixed to 30 and 0.90, respectively, and RVEA is set as the optimizer.
As we can see, when p f is set to 1 % and 3 %, the performance of FDD-MOEA does not change much and becomes slightly when p f is increased to 5% or 10%.This is also expected since the larger p f is, the less data will be available for model management on each client.

Conclusions and future work
It may become unrealistic to assume that all data can be made available for centralized training of surrogates in data-driven evolutionary optimization.To remove this strong assumption, we consider the challenging situation in which data are distributed on different local clients and propose a federated datadriven multi/many-objective evolutionary optimization algorithm in this work.By leveraging federated learning, a global surrogate is trained collaboratively for fitness approximations, a sorted averaging algorithm is used to aggregate the local models, and the uploaded local surrogates are utilized to provide the uncertainty of solutions.In addition, a modified acquisition function combining local and global information is proposed for surrogate management.Empirical results indicate that the proposed algorithm achieves comparable results on 49 benchmark problems with the state-of-the-art of centralized data-driven evolutionary algorithms.However, the proposed algorithm does not perform well on the problems with disconnected Pareto fronts, in which case the local model may strongly differ from each other, causing considerable model divergence.Hence, in the future, we will investigate more robust modelling techniques and new acquisition functions for handling strongly non-iid data for data-driven optimization.

Fig. 1
Fig. 1 An illustration of local training with server model.

Algorithm 2 :
Federated data-driven MOEA.Input : Dimension of the decision space d; Expensive real objective functions f ; Maximum iterations R; Maximum number of training samples l; Number of clients N ; Number of selected samples per round m; Failure probability of the communication p f .Output: Solutions X and their objective values Y X ← Latin hypercube sampling 11d − 1 points

Table 1
Test instances and their properties.

Table 2
Mean (std)of the IGD values obtained by FDD-MOEA, HeE-MOEA, GP-MOEA and K-RVEA for different decision dimensions when M = 3.

Table 3
Mean (std) of the IGD values obtained by FDD-MOEA, HeE-MOEA, GP-MOEA and K-RVEA for different numbers of objectives when d = 20.

Table 4
Mean (std)of the IGD values obtained by using different λ for different numbers of objectives when d = 30.

Table 5
Mean (std)of the IGD values obtained by using different p f for different numbers of objectives when d = 30.