Structural and Multidisciplinary Optimization

, Volume 46, Issue 2, pp 239–259

A modified NBI and NC method for the solution of N-multiobjective optimization problems


    • Department of Civil EngineeringUFPE
  • Silvana M. B. Afonso
    • Department of Civil EngineeringUFPE
  • Paulo R. M. Lyra
    • Department of Mechanical EngineeringUFPE
Research Paper

DOI: 10.1007/s00158-011-0729-5

Cite this article as:
de S. Motta, R., Afonso, S.M.B. & Lyra, P.R.M. Struct Multidisc Optim (2012) 46: 239. doi:10.1007/s00158-011-0729-5


Multiobjective optimization (MO) techniques allow a designer to model a specific problem considering a more realistic behavior, which commonly involves the satisfaction of several targets simultaneously. A fundamental concept, which is adopted in the multicriteria optimization task, is that of Pareto optimality. In this paper we test several well-known procedures to deal with multiobjective optimization problems (MOP) and propose a novel modified procedure that when applied to the existing Normal Boundary Intersection (NBI) method and Normal Constraint (NC) method for more than two objectives overcomes some of their deficiencies. For the three and four objective applications analyzed here, the proposed scheme presents the best performance both in terms of quality and efficiency to obtain a set of proper Pareto points, when compared to the analyzed existing approaches.


Multiobjective optimizationMultidimensional Pareto frontierNBI methodNC method

1 Introduction

The advance in computational and numerical capabilities allows more efficient engineering design, through the solution of optimization problems. Thus, new realistic and challenge optimization applications have been tackled. One of these, is the assumption that the true goal of a design is the improvement of a set of objectives. This formulation leads to the so-called multiobjective optimization problem.

A fundamental concept, which is adopted in the multicriteria optimization task, is that of Pareto optimality (Collette and Siarry 2004), in which the solution of the MOP is the set of points that have optimum trade-off relation. Several approaches proposed in literature to obtain Pareto points are implemented in this work. Among them we have methods like the NBI (Das and Dennis 1996) and NC (Messac et al. 2003), which are efficient approaches to obtain good Pareto distribution for two objective optimization problems (Macedo 2002; Bates 2003; Lalonde et al. 2009; Motta 2009). For problems involving more than two objective functions, both standard techniques may fail in covering the whole Pareto frontier.

To overcome such drawback a new modification of the above mentioned methods is presented in this work. The purpose is to obtain an even distribution of points over the whole Pareto frontier without significant additional computational cost. Other modification of standard methods, like that proposed by Messac and Mattson (2004) to NC method, is also examined.

A detailed description of the proposed modification is presented and the scheme is tested and compared to other existing schemes by the solution of several model problems. For the analyzed applications the results obtained validate the proposed scheme and demonstrate its superiority against the other tested schemes.

The next two sections briefly describe the MOP and the concept of Pareto optimality. After that, in Section 4, four methods proposed in literature are detailed, and the difficulties of them to handle problems involving more than two objectives are illustrated, especially for the NBI and NC methods. Also, in this section, a modification to them is proposed. Section 5 presents some examples of MOP with three and four objectives that illustrate the behavior of the results obtained by the analyzed methods, as well as those obtained by the proposed approach. The last section draws the main conclusions of present work.

2 Mathematical definition

The MO problem may be written as:
$$ \mathop {\min }\limits_{x\in\Re^{n}} \;F(x)\; = \left[ {f_1 (x),f_2 (x), \ldots ,f_{nobj} (x)} \right] \label{eq:ProDef1} $$
subject to the following constraints
$$ \begin{array}{@{}l} h_k (x)\; = \;0\;\quad k\; = \;1, \ldots ,ne \\ g_i (x)\; \le \;0\;\quad i\; = \;1, \ldots ,ni \\ x_{lj} \le x_j \le x_{uj} \;\quad j\; = \;1, \ldots ,n \\ \end{array} $$
in which x is the design variable vector, n is the dimension of x, F(x) is the set of objective functions, which is to be minimized, nobj is the number of objective functions, hk(x) is an equality constraint, gi(x) is an inequality constraint and xlj, xuj are, respectively, the lower and upper bounds of a typical design variable. Differently from the scalar optimization problems, in which a unique objective function is involved, MOP involves more than one objective and are also referred as vector optimization.

3 Pareto minima concept

Pareto optimality concept Collette and Siarry (2004) is used to obtain MO solutions. The Pareto minima, are points xp such that no other point x exists under the conditions:
$$ f_k(x)\; \le \;f_k(x^p),\quad k = 1,\ldots, nobj $$
$$ f_j(x)\; < \;f_j(x^p) $$
for one objective function fj at least.
Figure 1 shows a general MO problem in a bi-objective space in which the Pareto frontier is highlighted. The discussions about this concept can be found in detail elsewhere (Steuer 1985; Das and Dennis 1996; Messac et al. 2003; Collette and Siarry 2004; Messac and Mattson 2004; Arora et al. 2007).
Fig. 1

Design objective space for a bi-objective problem (Arora et al. 2007)

A single solution of problem described in (1) and (2) is frequently impossible to obtain because the objective functions in general conflict with each other. Using the Pareto concept, the designer has to identify as many Pareto minima points as possible. These points can be used to construct a point-wise approximation to the Pareto front.

4 Schemes for generating the Pareto front

There are several techniques to obtain the set of Pareto minima. In this work we will discuss the so-called Weighting Sum method (WS) (Steuer 1985), Min–Max method (Hwang et al. 1980), Normal Boundary Intersection method (NBI) (Das and Dennis 1996) and Normalized Normal-Constrain method (NC) (Messac et al. 2003). Currently, in literature, the latter two strategies are pointed out to have more success to obtain the Pareto curves (for bi-objective problems). The problems that arise from such methods, when more than two objectives are involved, are discussed in Section 4.5.

4.1 WS method

The WS method is the traditional approach considered in the MO framework. In this procedure, the original MOP (1 and 2) is converted into a single optimization problem through the consideration of a substitute objective function, subject to the set of constraints given in (2). The single substitute objective function takes the following format
$$ F^j \; = \;\left[ {\bf{\beta}^j } \right]^T \frac{f}{{f_0 }}\; = \;\sum\limits_{k = 1}^{nobj} {\beta _k^j \frac{{f_k }}{{f_{0k} }}} \label{eq:WSf} $$
in which f0k is the k objective function evaluated in the initial design x0 and the elements \(\beta _k^j \) are the weighting coefficients, that will also be used in all subsequent techniques discussed here. They represent the relative importance of each objective and are normalized according to:
$$ \sum\limits_{k = 1}^{nobj} {\beta _k^j \; = \;1} ,\ \beta _k^j \; \ge 0 \label{eq:betadef} $$

The Pareto optimum points are obtained solving many scalarizations of the MOP for various βj, j = 1 ...nPp, where nPp is the total number of Pareto points required.

This technique presents the drawback that an even spread of weights rarely produces an even spread of Pareto points and it is also unable to find solutions in non-convex regions. In certain cases, the difficulty to find Pareto minima points increases and the designer cannot have an estimation of the shape of the trade-off frontier. Recently, different adaptive WS methods have been proposed by several researchers (Kim and Weck 2004; Huang et al. 2008). Different type of weighted metric for the objectives can be considered as an alternative to improve the standard form of this scheme (Huang et al. 2008). Kim and Weck (2004) overcame the concave region problem and obtained uniformly spaced Pareto points through an adaptive weighted sum method that solves MOP with more than two objective functions. Their approach is not analyzed here.

4.2 Min–Max method

The Min–Max method was proposed aiming to overcome the drawbacks of the standard WS procedure. This methodology is able to obtain Pareto points in non-convex regions and improve the distribution of the Pareto points. In this procedure each objective function is transformed into another function \(\bar f_k \) normalized such as
$$ \bar f_k = \frac{{f_k - \min f_k }}{{\max f_k - \min f_k }}\ ,\ \ k = 1,...,nobj $$
The parameters “min fk” and “max fk” are respectively the minimum and maximum values of fk in the range of the scalar (or single objective) optimization solutions. The new vector of objective functions is
$$ \bar {\bf{F}} = \;\left[ {\bar f_1 ,\bar f_2 ,...,\bar f_k ,...,\bar f_{nobj} } \right] $$
A weighting factor βk is assigned to each objective function \(\bar f_k \) and the following optimization problem is carried out
$$ \begin{array}{@{}l} \mathop {\min }\limits_x \;\gamma (x) \\ {\text{subject to: }}\left\{ \begin{array}{@{}l} \text{the set of constraints def\/ined in (2), and} \\ \beta _k \bar f_k (x) \le \;\gamma (x),{\ }k = 1,...,nobj \\ \end{array} \right. \\ \end{array}\label{eq:MMPro} $$
$$ \gamma (x)\; = \;\max (\;\beta _k \bar f_k (x)),\begin{array}{*{20}c} {} & {k = 1,...,n} \\ \end{array}obj $$

4.3 The NBI method

The NBI procedure is based on the so called Das parameterization of the Pareto curve (Das and Dennis 1996) and produces an even spread Pareto points distribution. This property turns out to be very adequate for obtaining the trade-off solutions among the various conflicting objectives. The NBI procedure is briefly described in what follows, and the details of such scheme can be found in Das and Dennis (1996).

4.3.1 Basic definitions

The utopia point or shadow minima F* and the normalized vector of objective functions \({\bar {\bf F}}({\bf{x}})\) are defined respectively, as
$${\bf F}^* \; = \;[ f_1 ^* , \ldots ,f_{nobj}^* ]\; \rm{and\ } \text{${{\bar{\bf F}}}$}({\bf{x}}) = {\bf{F}}({\bf{x}}) - {\bf{F}}^* $$
in which each component \(f_i^*\) represents an individual local minima. Let \(\bf{x}_i^*\) be the respective minimizer of fi(x). The pay off matrix is defined as
$$ \Phi (i,j)\; = \;\left( {f_i ({\bf{x}}_j^* )\; - \;f_i^* } \right), \;\;i,j\; = \;1,\; \ldots ,\;nobj $$
Finally, the set \(\Re ^{nobj}\) of points that are convex combinations of Φ, i.e.
$$ \left\{ {{\Phi \beta} :\,{\beta} \in \Re ^{nobj} } \right\} $$
is referred to as the Convex Hull of Individual Minima (CHIM). In addition, \(\Im\) is defined to be the set of feasible objective points. The boundary of \(\Im\) is denoted \(\partial\Im\).

4.3.2 Generation of Pareto points

The efficient points in \(\Im\), which in most of the cases are Pareto points, are found by the intersection between the boundary \(\partial\Im\) and the normal emanating from the CHIM pointing towards the origin (Das and Dennis 1996), as illustrated in Fig. 2 for a bi-objective case.
Fig. 2

NBI method: graphical representation of the scheme (Das and Dennis 1996)

Denoting by \(\bar {\bf{n}}\) the unit normal to the CHIM, the set of points defined by \(\bf{p}\;=\;\boldsymbol{\Phi} \beta \) + \(t{\bar{\bf n}}\) lies on that normal. Then the intersection point between p and \(\partial\Im\) closest to the origin is the solution of the following sub problem:
$$ \begin{array}{@{}l} \mathop {\max }\limits_{x,t} \;t \\ \text{{subject to: }}\left\{ \begin{array}{@{}l} \text{the set of constraints def\/ined in (2), and} \\ \Phi \beta \; + \;t{\bar{\bf n}}\; \ge \;{\bar{\bf F}}(x) \\ \end{array} \right. \\ \end{array} $$

The original method uses the additional constrain as \(\Phi \beta + t{\bar{\bf n}}\; = \;{\bar{\bf F}}(x)\). The use of inequality form of (14), instead of the equality form, gives more freedom to the optimization algorithm to find global Pareto points and to avoid bad formulated optimization sub-problems, as will be discussed later in Section 4.5. The subproblem (14) is called the NBIβ subproblem. The parameters β provides an alternate parameterization of the Pareto set, which is called Das parameterization (Das and Dennis 1996). In this procedure an even spread distribution for β is used. This leads to an even distribution of the points ϕβ on the CHIM, called “CHIM Points”. The normal emanating from these evenly spaced points intersect the boundary of the set of feasible solutions, containing the Pareto optimal points. This entire process forces the arcs joining two consecutive Pareto points to have equal projections on the CHIM, hence the points obtained are uniformly spread in this sense. The whole concept extends for more than two objectives.

4.4 The NC method

The NC method was introduced by Messac et al. (2003) and attempt to improve the NBI method by reducing the number of non-Pareto points generated. These methods have been shown to be able to generate uniformly spread Pareto points. NC works in a similar fashion as the NBI method and its graphical representation can be seen in Figure 3, which illustrates the objective space for a general bi-objective optimization problem and the corresponding Pareto frontier.

The utopia line (or plane, similar to the CHIM in NBI method) is the line joining the two anchor points (or end points of the Pareto frontier). These anchor points are obtained when the generic ith objective is minimized independently. To obtain the Pareto points the utopia line is divided into several points \({\bar{\bf X}}_{p}\) (analogous to CHIM Points). In an interactive process, from each point \({\bar{\bf X}}_{p}\), a normal line (or plane) NU is used to reduce the feasible space as indicated in Fig. 3. Minimizing \(\bar f_2\) results in the Pareto point \(\bar{\bf F}^*\), consequently after translating the normal line for all \({\bar{\bf X}}_{p}\) points the whole set of Pareto solutions will be found.
Fig. 3

Graphical representation of the normal constraint method for bi-objective problems (Messac et al. 2003)

Mathematically, for each generated \({\bar{\bf X}}_{pj}\) point in the utopia line, the jth Pareto point is obtained solving the following problem
$${\mathop {{\rm{min}}}\limits_x } {\bar f_{nobj} } \left\{ \begin{array}{@{}l} \text{{subject to the set of constraints}}\\ \text{{defined in (2), and}} \\ \bar N_k \left( {{\bar{\bf F}} - {\bar{\bf X}}_{pj} } \right)^T \le 0,\ k = 1,...,nobj - 1 \\ \end{array} \right. \label{eq:NCpro} $$
in which \(\bar{\bf F}\) is the normalized objective function (8). Note that the solutions of this method depend on the objective function (\(\bar f_{nobj}\)) selected to be minimized. This aspect of the NC method will be elucidated later, in the example of Section 5.2. The directions \(\bar N_k \) are the utopia plane vectors, from \({\bar{\bf F}}(x^{k * } )\) to \({\bar{\bf F}}(x^{nobj * } )\), defined as
$$ \bar N_k = {\bar{\bf F}}(x^{nobj * } ) - {\bar{\bf F}}(x^{k * } ), \ {k = 1,2,...nobj - 1} $$
and the Hyperplane Point \({\bar{\bf X}}_{pj} \) is the jth point in the utopia plane which is obtained as
$${\bar{\bf X}}_{pj} = \sum\limits_{k = 1}^{nobj} {\beta _k^j {\bar{\bf F}}({\bf{x}}^{k * } )} $$

For more than two objectives, the projection of the utopia plane does not cover the entire Pareto front, thus Messac and Mattson (2004) provides an important improvement of the original NC method, to overcome this limitation. In addition, minimizing \(\bar f_{nobj}\) does not guarantee the intersection solution, between the hyper-planes NU and the Pareto front. The inequality constraints presented in (15), allow the solution to move in the Pareto front parallel to a hyperplane NU, i.e. in \(\bar N_k \left( {{\bar{\bf F}} - {\bar{\bf X}}_{pj} } \right)^T = 0\), for specific k and j. The limitations and some improvements of this method will be discussed later.

4.5 NBI and NC methods for multiple dimension Pareto front

Efficient Pareto points calculation over the whole frontier, for problems involving more than two objective functions, is still an important issue of investigation (Arora et al. 2007).

As already discussed, NBI Pareto points can be obtained trough the problem formulated in (14), once a β parametrization procedure (Das and Dennis 1996) has been selected. However, for MOP involving more than two objective functions, some of the components of β are not necessarily greater than zero (Das and Dennis 1996).

Consider Fig. 4a and b (Das and Dennis 1996), in which a three objective function space of a generic MOP is shown. In this figure the Pareto region is highlighted gray and the “CHIM points”, (\({{\boldsymbol{\Phi} \beta}}\)) (13) for NBI, or the \({\bar{\bf X}_{\bf p}}\) points (17) for NC method, are presented. These are the base points for the MO subproblems of both methods. Here the CHIM is represented by the triangular region composed by \({\bar{\bf F}}(x_1 ^*)\), \({\bar{\bf F}}(x_2 ^*)\) and \({\bar{\bf F}}(x_3 ^*)\).
Fig. 4

NBI (and NC) methods for three MOP (Das and Dennis 1996): a “CHIM Points” and b Illustration of CHIM + 

It can be noticed that it is possible to find Pareto points only inside the CHIM, which means that the obtained points do not cover the whole Pareto region. Moreover, the Pareto points could even be located outside the Pareto region, as can be seen in Fig. 4a, in which part of the projection of the segment \(\overline {{\bf{F}}(x_1^* )\ {\bf{F}}(x_3^* )}\) is outside the Pareto region. In this case, the original NBI method will find non Pareto points, due to its several equality constraints, imposed in the original problem formulation of such scheme. The different constraints imposed here (14) may overcame this problem. Similarly, this is ameliorated in NC method due to the existence of more flexible constraints in its formulation (15).

The modification proposed here requires an additional definition denoted CHIM + . CHIM +  represents the whole Pareto frontier projected at the hyper planes which contains the individual minima as indicated in Fig. 4b. It is worth to mention that CHIM +  is not a convex hull and for the two objective case CHIM +  = CHIM (Das and Dennis 1996).

Figure 4b illustrates also the untaken regions (hatched areas in the figure) by standard NBI and NC methods. In order to find points covering the whole Pareto frontier, it is necessary to distribute the base points at CHIM +  instead of at standard CHIM region, as illustrated in Fig. 4b. However, a priori CHIM +  is unknown.

To find the whole Pareto region, Messac and Mattson (2004) present a modification in the original NC method, overcoming the problem of unexplored regions. As illustrated in Fig. 5 the original utopia plane (analogous to the NBI CHIM concept) indicated by the smallest triangle is magnified in such a way that involves the whole Pareto front, which is a subset of the feasible set indicated in figure. From this new utopia plan (biggest triangle in the figure) the subregions which do not contain projection of Pareto points (white regions inside the outer triangle) are eliminated. Then the modified method can find the whole Pareto surface. However, the problem of optimizations in base points that do not project on the Pareto regions, become worse. In Fig. 5 it can be identified a very large amount of points in the darkest region that do not project into the feasible objective function space. The more non Pareto points found, the less efficient becomes the procedure, even applying a post-process filter procedure.
Fig. 5

Objective function space and the feasible domain of a three-MOP. Modification presented by Messac and Mattson (2004) to the original NC method

4.6 A modified approach

In this work, the proposed methodology aims to obtain the boundary of the Pareto front and as consequence the CHIM +  region, and to determine interior points inside the Pareto region. To achieve the Pareto front boundary, MOP involving subsets of the objective functions need to be solved. In this way each border of the Pareto frontier is obtained in turn.

During the revision of the present work, the paper Mueller-Gritschneider et al. (2009) was mentioned by a reviewer. In this paper a similar idea of our methodology is described. It is important to mention that the present approach has been developed since 2008, and the first results were published in Motta (2009). The major difference between the two works is the process of generation of base points (inner points), when more than two objectives are involved. In Mueller-Gritschneider et al. (2009) the distribution of the interior points is calculated by solving a linear programming optimization problem for each point, instead of employing the CVT DOE technique considered here. The CVT procedure may not lead to an exactly uniform distribution of the interior points, but as all the points are distributed simultaneously, the process become more efficient, when compared to the interior points generation process used in Mueller-Gritschneider et al. (2009). The methodology proposed here also uses auxiliary optimizations in the sub-problems, that improves the reliability of the procedure, as described in what follows. Moreover, numerical evidences, that the proposed methodology works for MOP involving more than three objective functions, are given in the present paper.

For example, consider a general three objective optimization problem, indicated in Fig. 6. Before building the whole Pareto surface, the borders \({\bf{F}}({\bf{x}}_{[1, 2]}^* )\), \({\bf{F}}({\bf{x}}_{[2, 3]}^* )\) and \({\bf{F}}({\bf{x}}_{[1, 3]}^* )\) are obtained. For instance, by solving a MOP considering only f1 and f2 objectives the Pareto border \({\bf{F}}({\bf{x}}_{[1, 2]}^* )\) is obtained, following by solving a MOP considering only f2 and f3 objectives the border \({\bf{F}}({\bf{x}}_{[2, 3]}^* )\) is obtained and similarly the border \({\bf{F}}({\bf{x}}_{[1, 3]}^* )\) is attained.
Fig. 6

Modified approach: Pareto front boundary and CHIM +  points

During this stage (bi-objective optimizations) the third objective (the ignored one) may assume a bad value (i.e. resulting in a non Pareto point), thus an additional (auxiliary) suboptimization must be evaluated. The latter is a scalar optimization problem considering the third objective function and the pair of functions previously optimized fixed as additional constraints. This feature will be detailed later.

In Fig. 6 an example of Pareto points distribution is shown, in which the black points were obtained as described before. The projection of such points define the CHIM +  region. This encompass the whole Pareto front, and the remaining CHIM +  points (highlighted in gray) are distributed inside (Motta 2009). The distribution of the base points on the interior of the CHIM +  region is obtained by an appropriate CVT (Centroidal Voronoi Tessellation) design of experiment (DoE) algorithm (Du et al. 1999; Giunta et al. 2003).

The CVT method is an iterative algorithm used to redistribute an initial design so that the distance between each points are optimized by achieving the configuration of minimum energy. To characterize the minimum energy, the centers of mass of the Voronoi Polygons must be determined to become the new generators. When the generators move to the centers of mass of the Voronoi regions, the relations between the new generators and each point are modified, and so the Voronoi Tessellation must be redesigned. This process must be repeated as many times as necessary for a satisfactory uniformity. There are many known methods for constructing CVT, in the present work an algorithm based on Du et al. (1999) is considered.

In the adapted algorithm used here, the projection of the Pareto points which form the boundaries of the CHIM +  region, where the interior points are distributed, are used to built the Voronoi Tessellation, but their positions are fixed during the CVT interactions.

Although the discussion carried out here, involves a three objective example, the procedure is, in principle, general and can deal with more than three objective functions. In this case, the basic idea is the same and can be summarized as follows:
  1. 1.

    Perform an optimization for each objective function individually;

  2. 2.

    Perform bi-objective optimizations for each possible bi-objective space;

  3. 3.

    Once the Pareto curves have been obtained, for all possible bi-objective combinations, solve the three objective problems for all possible permutation;

  4. 4.

    Once the Pareto surfaces have been obtained, for all combinations of three objective functions, conduct four objective problems for all possible permutation;

  5. 5.

    Subsequently continue with the above procedure until the total number of objective functions has been covered.


Applications considering the proposed methodology are presented in Section 5, for MOP involving three and four objectives.

The number of Pareto points obtained in each step, and consequently, the number of sub-optimizations performed, is determined so that the nPp remains the same as the originals methods (and equal to the desired nPp). For all the MO methods described here the number of optimization problems to be solved is equal to nPp plus the additional (auxiliary) scalar suboptimizations evaluated. In other words, the main difference between the MO techniques is on their optimization problem formulation.

Thus, for a MOP considering nobj objectives and k values of β a total of ((nobj + k − 2)!/(k − 1)!/(nobj − 1)! = nPp) sub problems are solved (Das and Dennis 1996). It is important to emphasize that when more objectives are considered more Pareto points are needed to cover the entire Pareto front properly.

For instance, in a four-MOP in which 5 values of β are considered ([0, .25, .5, .75, 1]), (following the parametrization presented in Das and Dennis 1996), a total of 35 (7!/4!/3!) sub-optimization problems should be solved, for all methods presented here. In the case of the modified version presented here, these 35 sub-problems are divided in:
  • Four scalar optimizations;

  • Six bi-MOP (each one computing three Pareto points) totalizing 18 sub-problems;

  • Four three-MOP (each one computing three Pareto points) totalizing 12 sub-problems;

  • and one four-MOP computing only one Pareto points, corresponding to vector; β = [.25, .25, .25, .25], the only vector β without a null component (that satisfy (6)).

Although the nPp used for all methods, in the applications analyzed here, be equal, the number of feasible solutions, significantly different, that satisfy the Pareto concept could vary for each method. Thus, the number of Pareto points effectively obtained (excluding overlapping points, non Pareto points and infeasible solutions) will be refer to nEPp (number of effective Pareto points), note that nEPp ≤ nPp.

The sub-optimizations in which any objective function is ignored may result in an optimum point that can move and get better in some of these omitted objective functions, without changing the value of the considered objective functions.

For example, consider the idealized problem involving two functions to be minimized, namely: \(f_1(x,y)=x^2\) and \(f_2(x,y)=y^2\). Clearly, the optimum and unique Pareto point is (x, y) = [0, 0]. Although, using the NNC or NBI methods, the result of the scalar optimizations (considering the objective functions individually), are deeply dependent on the starting point and so the remain of the MO process. The optimization of function f1 starting from any [x0, y0] point, could result in a (x, y) = [0, y0] point, or in a objective function space \((f_1,f_2)=[0,y_0^2]\). Thus, the result from the scalar optimization leads to non Pareto point (except if the starting point is [x0, 0]), the same observation is valid for function f2.

Therefore, after any sub-optimization in which any objective function is ignored, another auxiliary optimization is performed, starting from the optimum point obtained before. The objective function previously ignored is considered as a new goal and the optimum value achieved is preserved (or improved) as a result of an additional inequality constraint applied.

This process is introduced in the proposed methodology and all MOP solved in this paper, consider it in order to guarantee robustness. Note that for sub-optimizations in which any beta value is equal to zero the correspondent objective function is ignored by the WS and Min-Max methods, as well as in the methodology presented here. It is worth emphasizing that the majority of auxiliary optimizations considered during the solution of the examples of this paper, reach convergence very rapidly (less than three iterations).

The general algorithm of the proposed methodology is presented in the Table 1.
Table 1

General algorithm of the modified MO methodology

1.  Loop over objectives (for 1 to nobj)

     1.1  Run scalar optimization

     1.2  Run auxiliary optimization

           End loop

2.  Loop over dimension (for dim = 2 to nobj)

     2.1  Compute permutations

     2.2  Loop over permutations (for 1 to n_permutation)

           2.2.1  Update permutation data

           2.2.2  If (dim ≥ 3)

                  i.  Project Pareto boundary points

                ii.  Distribute interior points - CVT

                    End if

           2.2.3  Loop over interior points

                  i.  Run sub optimization

                ii  Run auxiliary optimization (if dim ≠ nobj)

                    End interior points loop (sub-optimization)

                End permutation loop

           End dimension loop

Remark 1

Step 1. Scalar optimizations and the auxiliary optimization cited (to guarantee robustness).

Step 2. Starting with bi-objective subproblems, than tri-objective subproblems and so on.

Step 2.1. For example, for the bi-objective subproblems of a problem with three objectives, the permutations computed are: [(f1,f2); (f2,f3);(f1,f3)].

Step 2.2. Obtain subsets of Pareto points for all permutations of the current dimension case (boundary subproblems if dimension≠nobj) . This can be consider as a subset of interior points for the current dimension, or a boundary subset for higher dimensions (if exist).

Step 2.2.1. Update the Pareto points (of the boundary) needed to obtain the interior points of the current permutation. For the example cited above, the first “permutation data” needed are the results of the one-objective optimization involving only f1 and only f2. But for the three dimension case (tri-objective subproblems) and permutation (f1,f2,f3), the “permutation data” needed are the results of the one- objective optimization involving f1 only, f2 only and f3 only, as well as the results (Pareto curves) of the bi-objective optimization (f1,2), (f2,3) and (f1,3).

Step 2.2.2. Project these (boundary) Pareto points (CHIM +  generation) and Generate the interior points via CVT.

Step 2.2.3. Obtain the sub optimization result for all interior points of the current permutation. The auxiliary optimizations here are analogous to that in step 1.2 (see remark step 1).

5 Numerical examples

In this section, several academic multiobjective applications, which consider three and four objective functions and a problem from the literature with three objectives, are studied to highlight some of the difficulties of the standard methods when dealing with this class of problems. The performance of the modified methods proposed here is also presented. Additionally to the examples presented here, an application on structural MOP with three objective functions, was successfully solved in Motta (2009) using the proposed modification combined with a reduced order modeling approach called Reduced Basis Method (Afonso et al. 2009). In the results of the present paper, the parameter nFC is the total number of function evaluations necessary for the Pareto solutions and nEPp is the number of effective Pareto points (described previously).

5.1 Three objective functions: first analytical test problem

Consider the following MO problem:
$$ \mathop {{\rm{min}}}\limits_x f(x)_i = x_i {\rm{, \ }}i = 1,2,3 $$
Subject to:
$$ \begin{array}{@{}l} x_1 \ge x_2^{-1} + x_3^{-1} \rm{;}\\ x_2 \ge x_1^{-1} + x_3^{-1} \rm{;}\\ x_3 \ge x_1^{-1} + x_2^{-1} \rm{;}\\ 0.2 \le x_i \le 10,{\ }i = 1,2,3 \\ \end{array} $$

This very simple problem results in a wide and smooth Pareto surface, that can show properly the untaken regions by the standard NBI and NC methods.

The solution to this problem using the standard methods are shown in Fig. 7a–d and the solution obtained by the modified versions proposed here are shown in Fig. 8a and b. The Pareto points were obtained considering 15 uniformly distributed βi values, with i = 1, ...3 which combined with each other and considering (6) leads to 120 different β vectors (Das and Dennis 1996) and so to 120 optimization subproblems. In both figures (for each method) the left hand side graphic shows the top views (f1(x) × f2(x)) of the objective functions space, where the gray scale refers to the value of the third objective function f3(x). The right hand side graphics present an isometric view of the three objective function spaces, the gray scale refers to the value of the third objective function f3(x) for a better visualization. For each method the caption of the figure presents, within the brackets, the nFC/nEPp parameters.
Fig. 7

Pareto points distribution for standard methods and the total number of function evaluations: First analytical test problem—left: top views; right: isometric view
Fig. 8

Pareto points distribution for the proposed modified methods and the total number of function evaluations: First analytical test problem—left: top views; right: isometric view

As can be observed, the WS and Min–Max methods lead to point concentration in some regions, empty portions in some other regions and the smallest number of function evaluations. WS method obtain just 22 effective Pareto points, out of the 120 desired. The NC and NBI methods give uniformly distributed points. However, they are unable to cover the whole Pareto surface, but solely the regions that result from the CHIM projection (i.e. the triangle formed by the individual minima). By analyzing the results obtained using the modified methods NBIm and NCm (which determine the boundary of the Pareto region), an even distributed set of points were obtained over the whole Pareto front, see Fig. 8. The standard methods, NBI and NC and their modified versions (NBIm and NCm), required basically the same number of function computations.

Figure 9 illustrates some aspects of the new procedure, where at Fig. 9a, a perspective view and at Fig. 9b, an isometric view, which is orthogonal to the projection plane (i.e. the individual minima plane), are presented. In these figures, the standard CHIM region is depicted by dashed lines. The black dots connected through lines represent the Pareto points obtained through the solution of two objective optimization subproblems. The projection of such points into the plane formed by the individual minima are represented by the “x” dots, defining the CHIM + . Finally, the internal points (in black) were obtained after determination of the CHIM + , using a CVT algorithm as described in Section 4.5. The presented results were obtained using NBIm, but those obtained using the NCm (not shown) were basically the same.
Fig. 9

The new methodology: Pareto boundary frontier (i.e. CHIM +  definition), internal points distribution and the standard CHIM region: a perspective view and b isometric view

5.2 Three objective functions: second analytical test problem

A second MO analytical model problem was considered and defined as:
$$ \mathop {{\rm{min}}}\limits_x f(x) = \left\{ \begin{array}{l} x_1 ^3 + x_2 + 2x_3 \\ x_1 + x_2 ^3 + 2x_3 \\ - x_1 x_2 x_3 \\ \end{array} \right\} $$
Subjected to:
$$ \begin{array}{@{}l} x_i \ge 0,{\ }i = 1,2,3 \\ x_1 ^2 + x_2 ^2 \le x_3 + 5 \\ 5(x_1 + x_2 ) \ge x_3 \\ \end{array} $$

For this particular example it can be observed that \(f_1 ^*=f_2 ^* \) , for \(x_1 ^*= x_2 ^*=(0,0,0)\). Due to that fact, the CHIM degenerates into a 3D straight line, and so, the Pareto points obtained through the NBI and NC methods can only achieve points over this line.

The results obtained using the methods implemented here are illustrated in Fig. 10, considering 120 Pareto points generated as described in Das and Dennis (1996). As before, the graphics show the top view (f1(x) × f2(x) ) of the objective functions space with the gray scale referring to the value of the third objective function f3(x). The parameters nFC/nEPp are given in the caption of each graphic.
Fig. 10

Pareto points distribution for each analyzed method and the total number of function evaluations: second analytical test problem

As indicated in Fig. 10a, the WS method lead to a poor result with basically just two distinct points. The Min-Max method lead to approximately the same result as the standard NBI and NNC methods. As mentioned, the CHIM degenerates into a 3D line, which can be seen in Fig. 10c and d, where the Pareto points were obtained through the NBI and NC method, respectively.

The proposed procedure led to a better distribution of the Pareto frontier for the NBIm case, as shown in Fig. 10e. The CHIM +  points have been obtained from the Pareto curves found as the solution of two objective subproblems. For this case the NCm (see Fig. 10f) was not able to give as good performance as the NBIm.

When using the NC (and NCm), in this case, the directions \(\bar N_k \) (16) (NC method) has only one independent vector, and so, the additional constraints of the method deteriorate in one plane. Due to the extra relaxation of the inequality constraints (see (15)) the NCm allows a larger solution mobility. Thus the solution fall to the center of the Pareto front, where f3 is minimized.

Finally, it should be said that the NCm results would be different if the sub-optimizations employed a different order for the objective functions then the adopted here, in which f3(x)  =   − x1x2x3 . For example, changing the objective functions order, if the last function (selected to be minimized) was \(f_3\; =\; x_1 ^3 + x_2 + 2x_3\) (previously f1), the internal solution should move to the point that minimize the function previously f1 (to the left), as can be seen in Fig. 10g. Figure 10g illustrates the results via NCm method in which the third function was changed by the first. This new solution should generate 78 internal points, but from those 47 solutions are non Pareto points (black “x” points). The NBIm solution is independent on the objective order.

5.3 Three objective functions: geometric test problem

Consider the problem defined in Fig. 11, which illustrates a two design variable, x1 and x2, space. A MOP is set taking three objective functions that are to minimize the distance of an arbitrary point x to the points A, B and C given in Fig. 11. Furthermore, consider that the feasible space exclude the circle presented in Fig. 11, i.e. the distance from x to point D has to be larger than 0.5.
Fig. 11

Geometric problem definition

By defining:
$$ {\bf{F}}\left( {\bf{x}} \right) = \left[ {\begin{array}{*{20}c} {\left\| {{\bf{x - A}}} \right\|} \\ {\left\| {{\bf{x - B}}} \right\|} \\ {\left\| {{\bf{x - C}}} \right\|} \\ \end{array}} \right] $$
A MOP can be formulated as:
$$ \mathop {{\rm{min}}}\limits_{\bf{x}} f_i ({\bf{x}}),\ i = 1,2,3 $$
Such that
$$ g({\bf{x}}) = \left\| {{\bf{x - D}}} \right\| - 0.5 > 0 $$
where F is the vector of the objective functions, g is the inequality constraint and x = (x1, x2) is the vector of design variables. The geometric problem can be helpful to understand and visualize the result of the MOP.
Figure 12a presents, in gray, the exact Pareto set. Figure 12b–g present the Pareto solutions that result from the application of different methods investigated. The parameters nFC/nEPp for each method are given in the caption within the brackets.
Fig. 12

Pareto solutions for the geometric MOP and the total number of function evaluations

From these results, it should be emphasized the existence of empty regions (i.e. non even distribution of Pareto points) and a very large number of function evaluations when WS scheme (Fig. 12b) is adopted. When comparing to WS, the Min–Max method (Fig. 12c) leads to a better result determining the boundary of the optima region properly, with less function computations, but still with a non uniform distribution of both boundary and interior points.

The use of NBI (Fig. 12d) and NC method (Fig. 12e) result in good distribution of the Pareto solutions. However, NC scheme gives infeasible solutions (dots depicted by black “x” in Fig. 12e) and several points that do not satisfy the Pareto criteria. Furthermore, the original NBI method (which uses the equality constraint instead of the adopted inequality (14)) may present non Pareto points and infeasible solutions. The inequality constraint considered in the formulation of the NBI method (14)) allows the solution (related to CHIM points with no projection of the Pareto front) to move to the Pareto front, see Fig. 12d.

When using the proposed modification, i.e. NBIm and NCm, a very good Pareto points distribution were obtained (see Fig. 12f and g), covering the whole exact Pareto set with a well defined boundary and an even distribution of points. Moreover, no unfeasible and non Pareto solution was obtained. Finally, it must be stressed that all that were obtained with a smaller number of function evaluations when compared with the other analyzed methods.

5.4 Four objective functions: analytical test problem

This is an extension of the problem presented in Section 5.1, to a four dimension space. This problem can be defined as follows:
$$ \mathop {{\rm{min}}}\limits_x f(x)_i = x_i {\rm{, \ }}i = 1,2,3,4 $$
Subject to:
$$ \begin{array}{@{}l} x_1 \ge x_2^{-1} + x_3^{-1} + x_4^{-1}; \\ x_2 \ge x_1^{-1} + x_3^{-1} + x_4^{-1}; \\ x_3 \ge x_1^{-1} + x_2^{-1} + x_4^{-1}; \\ x_4 \ge x_1^{-1} + x_2^{-1} + x_3^{-1}; \\ 0.2 \le x_i \le 10,\;i = 1,2,3,4 \\ \end{array} $$
This simple problem is proposed here to test the robustness of the methods when dealing with a high dimensional MOP. The solution to this problem using the standard methods are shown in Fig. 13a–d, and the results obtained using the modified versions proposed here are shown in Fig. 14a and b. In both figures the Pareto points were obtained considering ten uniformly distributed βi values, with i = 1, ...4 which combined with each other as suggested in Das and Dennis (1996) leads to 220 different β vectors and, consequently, to 220 optimization subproblems.
Fig. 13

Pareto points distribution for each standard method and the total number of function evaluations: Analytical test problem with four objective functions
Fig. 14

Pareto points distribution for the modified version of the NBI and NC methods and the total number of function evaluations: Analytical test problem with four objective functions

For each method, the left hand side graphics show the top views (f1(x) × f2(x)) of the objective functions space. The right hand side graphics present an isometric view for the first three objective function spaces. The graphics in the center illustrate a perspective view of the first three objective function spaces. In all figures the gray scale refers to the value of the fourth objective function f4(x). For each method the caption of the figure presents, within the brackets, the parameters nFC/nEPp.

As can be observed, the WS and Min-Max methods lead to points concentration in some regions and empty portions in some other regions. The NC and NBI method give uniformly distributed points. However they are unable to cover the whole Pareto front, but solely the region that result from the CHIM projection (i.e. the hyperplane formed by the individual minima).

By analyzing the results obtained using the modified methods NBIm and NCm, which determine the boundary (curves and surfaces) of the Pareto region, an even distributed set of points were obtained over the whole Pareto region, with approximately the same number of function evaluations than the respective standard methods.

5.5 Gear box design

The optimization problem of a gear box design (Kupapatz and Azarm 2001) is now considered. In Huang et al. (2006) the problem is reformulated as a tri-objective optimization problem, in which two constraints of the original scalar optimization problem is considered as two new objectives. This version of the problem will be studied here. This problem can be formulated as follows:
$$ \mathop {{\mathop{\rm Min}\nolimits} }\limits_{x \in {{\bf{R}}^7}} {\bf{F}}(x) = \left[ {{f_1}(x),{f_2}(x),{f_3}(x),} \right] $$
subject to:
$$ \begin{array}{@{}l} 27{x_1}^{ - 1}{x_2}^{ - 2}{x_3}^{ - 3} \le 1,\\ 397.5{x_1}^{ - 1}{x_2}^{ - 2}{x_3}^{ - 2} \le 1,\\ 1.93{x_2}^{ - 1}{x_3}^{ - 1}{x_4}^3{x_6}^{ - 4} \le 1,\\ 1.93{x_2}^{ - 1}{x_3}^{ - 1}{x_5}^3{x_7}^{ - 4} \le 1,\\ {{{x_2}{x_3}} \mathord{\left/ {\vphantom {{{x_2}{x_3}} {40}}} \right. \kern-\nulldelimiterspace} {40}} \le 1,\\ {{{x_1}{x_2}^{ - 1}} \mathord{\left/ {\vphantom {{{x_1}{x_2}^{ - 1}} {12}}} \right. \kern-\nulldelimiterspace} {12}} \le 1,\\ {{{x_1}{x_2}^{ - 1}} \mathord{\left/ {\vphantom {{{x_1}{x_2}^{ - 1}} 5}} \right. \kern-\nulldelimiterspace} 5} \ge 1,\\ 1.9 - {x_4} + 1.5{x_6} \le 0,\\ 1.9 - {x_5} + 1.1{x_7} \le 0,\\ {{A1} \mathord{\left/ {\vphantom {{A1} {\left( {1300B1} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {1300B1} \right)}} \le 1,\\ {{A2} \mathord{\left/ {\vphantom {{A2} {\left( {850B2} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {850B2} \right)}} \le 1, \end{array} $$
$$ \begin{aligned} {f_1}(x) = &+ 0.7854{x_1}{x_2}^2(10{x_3}^2/3 + 14.9334{x_3} - 43.0934)\\[3pt] & - 1.508{x_1}({x_6}^2 + {x_7}^2) + 7.4777({x_6}^3 + {x_7}^3)\\[3pt] & + 0.7854({x_4}{x_6}^2 + {x_5}{x_7}^2); \end{aligned} $$
$$ f_2(x) = A1/B1; $$
$$ f_3(x) = A2/B2; $$
$$ A1 = \sqrt {({{(745{x_4}{x_2}^{ - 1}{x_3}^{ - 1})}^2} + 1.69 \times {{10}^7})} , $$
$$ B1 = 0.1{x_6}^3, $$
$$ A2 = \sqrt {({{(745{x_5}{x_2}^{ - 1}{x_3}^{ - 1})}^2} + 1.575 \times {{10}^8})} , $$
$$ B2 = 0.1{x_7}^3 $$
The three objectives are:
  • Minimize the speed reducer volume.

  • Minimize the stress in one shaft.

  • Minimize the stress in second shaft.

The problem is subjected to a number of constraints imposed to the design variables limits and by design practices, as: stress of teeth, transverse displacement of shafts, generated torque and the stress of shafts itself (used in objective). For more details about the problem, refer to Kupapatz and Azarm (2001), Huang et al. (2006) and Sanchis et al. (2008).

A modified version of the NC (NNC) method called enhanced normalized normal constraint (ENNC) method was applied to the tri-objective problem in Sanchis et al. (2008). As the example 2 (Section 5.2) only two different anchor points were obtained by the scalar optimizations. In Sanchis et al. (2008) three different pseudo anchor points were built (by assuming the worst value of the objective functions which are not minimized) to expand the Utopia hyperplane (CHIM) in order to obtain a more wide region of the Pareto surface. The average objective function evaluations was 66 for each of the 1,275 runs (points) of the SQP algorithm, totalizing 84,150 function evaluations.

The results obtained using all methods analyzed here are illustrated in Figs. 15 (standard methods) and 16 (proposed methodologies), with the nFC/nEPp (within the brackets) in the caption of each technique. A top view and an isometric view of the Pareto points in the space of the objective functions are illustrated, where the gray scale refers to the value of the third objective function f3(x).
Fig. 15

Pareto points distribution for standard methods and the total number of function evaluations: gear box problem
Fig. 16

Pareto points distribution for the proposed modified methods and the total number of function evaluations: box gear problem

As the other examples, the WS method obtains a poor result. The Min-Max method performs slightly better when compared to WS method.

Similarly to the example 2, the CHIM degenerates to a 3D line, deteriorating, consequently, the results by the standard NBI and NNC methods. The NNCm also, obtained results that follows the behavior presented in the example 2, in which the internal Pareto points were concentrated in a 3D curve.

Finally, by using the NBIm procedure a well distributed Pareto points were obtained, leading to the best performance among the methods presented here and the average objective function evaluations was 66, same as the results from the work Sanchis et al. (2008), cited before.

5.6 Results summary

The summary of the performance of each method for all MOP tested here is presented in Table 2. In that table the “Time (s)” indicates the total CPU time to obtain the MO solution, FC/nEPp indicates the number of function evaluations FC per effective (excluding overlapping points) Pareto points obtained nEPp, nnPp indicates the number of non Pareto points obtained including infeasible solutions. In the brackets of each example label the required nPp is shown. Finally, the parameter Evness, that appears in Table 2, indicates the quality of the distribution of the points, the closer to zero the better.
Table 2

Performance of studied cases


Time (s)


\(\emph{nnPp}^{\rm a}\)



Example 1 (120)





































Example 2 (120)





































Example 3 (120)





































Example 4 (224)





































Example 5 (120)





































aNumber of non-Pareto points obtained

The uniform distribution of the Pareto points (Evness) was calculated considering a procedure presented by Messac and Mattson (2004) in which two distances related to each point are calculated, called dlow and dup. The parameter dlow refers to the minimum distance of each point to another. The parameter dup states the maximum radius of a sphere touching each point, with no other points inside the sphere. The Evness parameter is calculated as follows:
$$ ev = \sigma (\bf{D})/\mu (\bf{D}) $$
in which
$$ \bf{D} = \left[ {\bf{d}_{\rm low}; \bf{d}_{\rm up} } \right] $$
and σ is the standard deviation and μ is the arithmetic mean of D.

In examples 1–3 and 5 a CVT algorithm was used to obtain the distribution of 78 interior points in a bi-dimensional space. The CPU time consumed by the CVT algorithm in these cases was around 3.5 s. Note that the points in the three dimensional space are projected in a plane, so the CVT algorithm is performed in a bi-dimensional space.

For the example 4 (four-MOP) it was necessary four CVT runs each one to distribute a set of 28 interior points in a bi-dimensional space and one run for 60 interior points in a three dimensional space. The CPU time consumed by the CVT for each set of 28 interior points (in a bi-dimensional space) was around 1.7 s. The CPU time consumed by the CVT, for distribute 60 interior points (in a three dimensional space), was around 26 s. Finally, for example 4 the total CPU time used for the distribution of interior points via CVT was approximately 33 s.

Note that those CVT timing are independent of the functions involved in the problem. Thus, for MOP considering (objective or constraint) functions that are computationally more expensive, like problems involving numerical simulation, the CVT time become negligible.

For all analyzed cases, despite the wide solution and the Pareto Frontier quality, the NBIm maintain (or improve) the computational efficiency, as the number of function evaluations per Pareto point is approximately the same as the standard methods.

In the examples 2 and 5 two objective functions has the same solution. Thus, by using the modified procedure presented here, the solution of the Pareto curves (bi-MOP) related to this two functions degenerates in one point. So the 15 Pareto points of this curve generates in only 1 effective Pareto point, that is why in those two examples, the NBIm does not obtain the total nPp (nEPp  =  nPp − 14). Note that this feature is predictable and do not reduce the efficiency of the method. As a matter of fact, if two function has the same solution individually, the Pareto points between those are not calculated.

Although the smaller number of Evness of the standards NBI and NNC methods, it has been shown that these methods do not cover the entirely Pareto Front, as the NBIm one. Finally, the uniform distribution of the interior points might be improved, by using other distribution method instead of the CVT algorithm adopted here.

6 Conclusions

In this paper several multiobjective optimization techniques, namely WS, Min–Max, NBI and NC, were implemented and tested, giving results that are in agreement with literature. It was observed that when more than two objective functions were considered NBI and NC were not able to capture properly the Pareto frontier and they can generate bad formulated optimization subproblems which lead to a large number of unnecessary function evaluations. Also, in some cases, they capture infeasible solutions for the Pareto frontier. In order to eliminate such bad behaviors a modification to these methods was proposed here and referred as NBIm and NCm.

Four examples, each one of them involving three objective functions each, and one example with four objective functions were analyzed. The results obtained using the developed methodology were compared with the existing methods. For all analyzed cases NBIm produces the best solution in terms of Pareto Frontier quality and maintaining (or improving) the computational efficiency, as the number of function evaluations per effective Pareto points are approximately the same when such technique is used. Further, no infeasible solution was obtained.

The methodology proposed is general for any number of objective functions and it is expected to behaves satisfactorily, as demonstrated for three and four objective functions.


The authors acknowledges the financial support given by the Brazilian research agency CNPq and Pernambuco state research agency FACEPE to the execution of the present work.

The authors acknowledges too, all the reviewers, which comments help to improve the current paper. It was particularly important to our knowledge the suggestion, during the second round of revision, of the paper Mueller-Gritschneider et al. (2009) in which a similar methodology to the one presented here was proposed.

Copyright information

© Springer-Verlag 2011