1 Introduction

In multiple attribute decision making (MADM) the ’best’ alternative is chosen from a set of alternatives based on the values of their attributes. MADM is applied to very diverse problems, it is used as a tool in the field of economics as well as in the manufacturing environment Venkata Rao (2007) and Greco et al. (2016). Attributes can be divided into cost and benefit attributes, where cost attributes are attributes one wants to be as low as possible, and benefit attributes one wants to be as high as possible. The choice of the ’best’ alternative is mainly based on ranking the alternatives. A preference relation compares the alternatives according to their attributes and assigns a well-defined degree to each alternative on an appropriate scale. In an economical decision making process both cost and benefit attributes are mostly expressed as the number of units of an underlying currency. In other fields of application attributes represent performance measures or metrics, and the attributes might have a dimension and can be expresses in different units.

In this paper we propose a multiple attribute relative quality measure as a preference relation on a set of alternatives. The fact that cost attributes and benefit attributes might be positively or negatively correlated raised the question whether the ranking of alternatives by a relative quality measure will differ from those performed by other MADM methods and if the difference might due to the effect of correlated attributes. The relative method proposed can be related to the method called data envelopment analysis (DEA), a non-parametric technique widely employed in economics, aimed to determine the relative efficiency of decision making units Cooper et al. (2011), defined as a ratio of weighted sum of outputs to a weighted sum of inputs. Like DEA the proposed procedure compares the attributes of each alternative with those of the other alternatives. This is done by means of the arithmetic mean of the relative benefit attributes and the harmonic mean of the relative cost attributes. Both harmonic and arithmetic mean are related, as the harmonic mean is the reciprocal dual of the arithmetic mean. It is this property that makes a relation with DEA possible. By the method proposed an alternative having the maximum relative quality of one can be considered as an extreme efficient unit according to DEA. In case all alternatives are distinct only a single alternative might be ranked with the maximum relative quality of one (see Sect. 2.3 for the definition of distinct). The comparison by means of the harmonic and arithmetic mean performs a kind of implicit weighting, where an attribute is weighted by the corresponding attribute of another alternative. This implicit weighting becomes more explicit in how the method relates to DEA (Sect. 2.4). A simple use case of a wide area network (WAN) will be used to illustrate the method.

Finally, the method of the presented relative quality measure is compared to three other MADM methods that apply Simple Additive Weighting, namely, the MADM methods based on OWA operator, maximizing deviations, and information entropy, and compared to the PROMETHEE II method. A problem with a large decision matrix is chosen to illustrate how the methods agree and how they disagree in ranking the alternatives.

2 Relative quality measure

In this section we present a relative quality measure based on the arithmetic and harmonic mean. To illustrate the concept of this measure we will make use of a small use case example of a wide area network (WAN). The use case of a WAN is chosen as an example where the attributes represent different performance measures, each expressed in its own units.

2.1 WAN use case example

To have some understanding of the different attributes and their values applied in the WAN use case, we give a short explanation. Applying different energy efficient strategies to reduce the power in a network will result in different alternatives of the network. With an energy efficient strategy the network owner of the network wants to reduce the total power dissipation P of the equipment in the network, a cost attribute. However, applying an energy efficient strategy may have side effects and it is good practice to consider the total power dissipation P in the network with respect to other performance measures of the network. Another cost attribute is the power usage effectiveness (PUE) (https://en.wikipedia.org/wiki/Power_usage_effectiveness. Accessed June 2016), it measures how efficiently a site where network equipment is housed uses energy, it is the quotient of the total power used for the housing of the equipment divided by the power of the network equipment. The more network equipment in a small housing, the more energy is needed for cooling, resulting in a larger value for the PUE. The PUE is a dimensionless measures with values in practice between 1 and 3. A third cost attribute applied in the use case is the mean latency L of the network. It measures the mean time it takes for data to be transported through the network. The network owner wants this value to be as low as possible, because users of the network appreciate quick data transport with the lowest delay as possible.

Beside these three cost attributes there are two benefit attributes in the use case. A benefit attribute is a performance measure which values the network owner wants to be as high as possible. The two benefit attributes are the mean utilization U of the network equipment, and the reliability R of the network. If only a small part of the capacity of a network device, switch or router, is used, the dissipated power by the device is to a large extent wasted power. So a network owner wants to keep the equipment busy by, e.g., data routing strategies. Powering down underutilized equipment and rerouting data will increase the utilization U, and lower the total power consumption P of the network, but this will also have a negative side effect on another benefit attribute, the reliability R of the network, and the mean latency L of the network might be affected. The smaller the number of possible routes available in the network to send data from site A to B, the lesser the reliability R of the network. Because, if equipment fails or becomes malfunctioning, the network owner will need alternative routes through the network to bypass the network part with the broken equipment.

With this small background of the performance measures in the use case, and their presented values listed in Table 1, the dilemma of the decision problem becomes clear. Focusing only on the total power dissipation P of the network might not lead to a desirable solution.

Table 1 Performance measures of three alternatives of a hypothetical WAN
Full size table

Suppose alternative 1 in Table 1 represents the current network and alternatives 2 and 3 what the network might be after applying a specific energy efficient strategy, e.g., powering down nodes in the network. Looking at the better power dissipation of alternative 3 one might draw the conclusion that the energy efficiency of the network can be increased by 31%. But if traffic is relayed differently for alternative 2 and 3, because equipment is switched off, less different paths are available. In such a situation the reliability may drop, as in a reliable network any origin-destination pair should have a primary and a secondary path. It might also happen that the average PUE becomes larger, as switched off equipment happens to be at sites with a relatively lower PUE. Also the average latency in the network may increase by powering down nodes. In this example the minimal value for a cost attribute like power consumption, 750 W, will not occur simultaneously with the maximal value of a benefit attribute like reliability, 98%.

2.2 Relative quality measure

To arrive at a relative quality we start with comparing each alternative out of n alternatives with each other, in such a way that the units of the attributes are of no influence on the comparison of these alternatives.

Consider a set of n network alternatives \(\{N(j)\},\,j=1,\ldots ,n\), each alternative j having (multiple) cost attributes \(x_{ij}, i=1,\ldots ,m\) and (multiple) benefit attributes \(y_{rj}, r=1,\ldots ,s\). The comparison of network alternative j with respect to network alternative o is defined as:

$$\begin{aligned} q_{j;o} = \left( \frac{1}{m} \sum _{i=1}^m \frac{x_{io}}{x_{ij}} \right) \times \left( \frac{1}{s} \sum _{r=1}^s \frac{y_{rj}}{y_{ro}} \right) \end{aligned}$$
(1)

The first factor is the mean of ratios of different type of cost attributes, where each ratio compares a cost attribute of alternative o with the corresponding cost attribute of network alternative j. The second factor is the mean of ratios of different type of benefit attributes, where each ratio compares a benefit attribute of network alternative j with the corresponding benefit attribute of alternative o. Suppose alternative o has cost and benefit attributes worse than any other alternative j, then the first factor of \(q_{j;o}\) will be larger than 1, and the second factor will also be larger than 1, for \(j\ne o\). So the larger the values for \(q_{j;o}\ j=1,\ldots n, j\ne o\), the smaller the relative quality of alternative o needs to be. It is clear that the comparison defined by \(q_{j;o}\) is invariant under a change of units the attributes are expressed in, e.g., Watt instead of kWatt.

Another way to write \(q_{j;o}\) is by considering two vectors

$$\begin{aligned} {\mathbf{x}}_{j;o}=\left( \frac{x_{1j}}{x_{1o}}, ...,\frac{x_{mj}}{x_{mo}}\right) ,\ {\mathbf{y}}_{j;o}=\left( \frac{y_{1j}}{y_{1o}}, ...,\frac{y_{sj}}{y_{so}}\right) \end{aligned}$$
(2)

where \({\mathbf{x}}_{j;o}\) is a vector of cost attributes of alternative j, each weighted by the corresponding cost attribute of alternative o, and vector \({\mathbf{y}}_{j;o}\) is a vector of benefit attributes of alternative j, each weighted by the corresponding benefit attribute of alternative o. Now \(q_{j;o}\) can be written as

$$\begin{aligned} q_{j;o}=\frac{A({\mathbf{y}}_{j;o})}{H({\mathbf{x}}_{j;o})} \end{aligned}$$
(3)

the quotient of the arithmetic mean A of weighted benefit attributes of alternative j with respect to alternative o, and H the harmonic mean of weighted cost attributes of alternative j with respect to alternative o. The harmonic mean is defined by:

$$\begin{aligned} H({\mathbf{x}})=\frac{m}{1/(x_{1;o})+\cdots +1/(x_{m;o})} \end{aligned}$$
(4)

The harmonic mean has the property that it is small if any of its values is also small, whereas the arithmetic mean shows the inverse behavior. By defining \(q_{j;o}\) according to Eq. (3) both the behavior of the arithmetic and harmonic mean are in agreement with each other, i.e., \(q_{j;o}\) is large if any of the cost attributes of alternative j is small compared to the corresponding cost attribute of alternative o, because H becomes small, and \(q_{j;o}\) is large if any of the benefit attributes of alternative j is large compared to the corresponding benefit attribute of alternative o. In other words, if alternative j outperforms alternative o in some of its attributes the larger \(q_{j;o}\) will be. So the relative quality of alternative o should be inversely related to the maximal value of the values \(q_{j;o}\ j=1,\ldots n\).

We resort to the use case example in Table 1 to illustrate how to arrive at a relative quality measure for each alternative. According to this example a network alternative j has 3 cost attributes \(x_{1j}\), \(x_{2j}\), \(x_{3j}\) and 2 benefit attributes \(y_{1j}\), \(y_{2j}\), or \(N(j)=(x_{1j},x_{2j},x_{3j},y_{1j},y_{2j})=(P_j,L_j,PUE_j,U_j, R_j)\). The comparison of alternative j with respect to alternative o becomes:

$$\begin{aligned} q_{j;o} = \left( \frac{1}{3} \left( \frac{P_o}{P_j} +\frac{L_o}{L_j} + \frac{PUE_o}{PUE_j} \right) \right) \times \left( \frac{1}{2} \left( \frac{U_j}{U_o} + \frac{R_j}{R_o} \right) \right) \end{aligned}$$
(5)

We proceed to construct a matrix Q with entries \(Q(o,j) = q_{j;o}\):

$$\begin{aligned} Q(o,j) = \left( \frac{1}{m} \sum _{i=1}^{m} \frac{x_{io}}{x_{ij}} \right) \times \left( \frac{1}{s} \sum _{r=1}^{s} \frac{y_{rj}}{y_{ro}} \right) \end{aligned}$$
(6)

so a row Q(o) contains all the comparisons of alternatives j with alternative o. In Table 2 the entries of this matrix Q are listed for the use case example above.

Table 2 Matrix Q according to Eq. (6) to determine the relative qualities for the use case example in Table 1
Full size table

Let’s look at the largest value of each row Q(o), i.e., \(max(q_{j;o})\ j=1,\ldots n\). If \(max(q_{j;o})>1\) then alternative o is outperformed by alternative j. The simplest way to arrive at a relative quality for alternative o is to take the inverse of \(max(q_{j;o}),\ j=1,\dots ,n\). So for the use case example the network alternatives 1, 2 and 3 have a relative quality \(q_1=1/1.2935=0.7730\), \(q_2=1\) and \(q_3=1/1.0780=0.9276\), respectively. This yields the following ranking of the alternatives: \(2\succ 3\succ 1\).

In this example we can notice that all entries in row \(Q(2,j),\ j\ne 2\), are smaller than one. This illustrates a special case with alternative 2 the only alternative having a relative quality equals 1, and necessarily all other alternatives having a lower relative quality. This we will be illustrated in the next section.

2.3 On the ordering defined by the relative quality

Combining Eqs. (2), (3) and (6), a relation between an entry Q(o, j) and its transpose entry Q(j, o) of matrix Q becomes clear. According to definition (2) it holds \((x_{j;o})_i=1/(x_{o;j})_i\) and \((y_{j;o})_i=1/(y_{o;j})_i\). Because the harmonic mean H is the reciprocal dual of the arithmetic mean A, i.e., \(1/H(1/x_1,\dots ,1/x_m)=A(x_1,\dots ,x_m)\), and \(A\ge H\), we can write

$$\begin{aligned} {Q(o,j)=\frac{A({\mathbf{y}}_{j;o})}{H({\mathbf{x}}_{j;o})}=\frac{A({\mathbf{x}}_{o;j})}{H({\mathbf{y}}_{o;j})}\ge \frac{H({\mathbf{x}}_{o;j})}{A({\mathbf{y}}_{o;j})}=\frac{1}{Q(j,o)} } \end{aligned}$$

So for any two entries Q(o, j) and Q(j, o) with \(o\ne j\) it holds:

$$\begin{aligned} {Q(o,j)<1 \ \rightarrow \ Q(j,o)>1, \quad o\ne j} \end{aligned}$$

It can also be that \(Q(j,o)=Q(o,j)=1\), then by the reciprocal dual property we get

$$\begin{aligned} 1=\frac{A({\mathbf{y}}_{j;o})}{H({\mathbf{x}}_{j;o})} = \frac{A({\mathbf{x}}_{o;j})}{H({\mathbf{y}}_{o;j})}\ge \frac{H({\mathbf{x}}_{o;j})}{A({\mathbf{y}}_{o;j})}=1 \end{aligned}$$

So the equal sign must hold, and we arrive at

$$\begin{aligned} {1=\frac{A({\mathbf{x}}_{o;j})}{H({\mathbf{y}}_{o;j})} =\frac{H({\mathbf{x}}_{o;j})}{A({\mathbf{y}}_{o;j})}} \end{aligned}$$

If \(A({\mathbf{x}}_{o;j})>H({\mathbf{x}}_{o;j})\) then also \(H({\mathbf{y}}_{o;j})>A({\mathbf{y}}_{o;j})\), which cannot be the case. So \(A({\mathbf{x}}_{o;j})=H({\mathbf{x}}_{o;j})\) and \(H({\mathbf{y}}_{o;j})=A({\mathbf{y}}_{o;j})\), and consequently all \((x_{o;j})_i\) are the same and all \((y_{o;j})_i\) are the same, as \(A({\mathbf{x}})=H({\mathbf{x}})\) if and only if all \(x_i\) are the same. An example of two alternatives, each having two cost and two benefit attributes, for which holds \(Q(j,o)=Q(o,j)=1\), is \(o: \{6,6,9,9\}\) and \(j: \{2,2,3,3\}\). In that case row Q(o) and row Q(j) have both two entries equals 1, Q(o, j), Q(o, o), and Q(j, j), Q(j, o). Futhermore, all other entries are the same \(Q(o,k)=Q(j,k)\), i.e., alternative o and j are ranked the same, they are not distinct.

So if there exists a row in matrix Q with entries \(Q(o,j)<1\) for \(j\ne o\), then it is the only row with this property and the relative quality \(q_o=1\), as the maximum of its row is given by \(Q(o,o)=1\).

It can also happen that every row of Q has at least one entry \(Q(o,j)>1\) for \(j\ne o\). Consider the set of alternatives in Table 3, each alternative having three cost attributes and three benefit attributes:

Table 3 Set of 6 alternatives, each alternative having 3 cost attributes C1(−), C2(−) and C3(−) and three benefit attributes C4(+), C5(+) and C6(+)
Full size table

In this example alternative A5 has cost attributes twice as high as those of alternative A6, but its benefit attributes are also twice as high, so both alternatives will have the same row in Q and consequently will be ranked the same. The corresponding matrix Q(o, j) can be seen in Table 4. We see that every row of Q has at least one entry larger than 1. The ordering is defined by the relative qualities \(q_1=1/1.577=0.6341\), \(q_2=1/1.197=0.8354\), \(q_3=1/1.393=0.7178\), \(q_4=1/2.658=0.3763\), \(q_5=q_6=1/1.279=0.7818\), resulting in: \(A2\succ A5,A6\succ A3\succ A1\succ A4\).

Table 4 The matirx Q for the 6 alternatives in Table 3
Full size table

2.4 Relation to data envelopment analysis

A widely used method to determine a relative efficiency, originating from the field of economics, is data envelopment analysis (DEA), introduced by Charnes, Cooper and Rhodes, an approach to judge the performance of so called decision making units (DMU). A review of the history and the application of this method can be found in Cooper et al. (2011). According to this method for a set of DMUs, \(\{DMU(j)\}, j=1,\dots ,n\), each DMU(j) having (multiple) inputs \(x_{ij}\), \(i=1,\ldots ,m\) and (multiple) outputs \(y_{rj}\), \(r=1,\ldots s\), the technical efficiency \(z_o\) of a DMU(o) is defined as a linear program problem:

$$\begin{aligned} z_o = max(z),\quad z = \sum _{r=1}^{s} \mu _{ro} y_{ro} \end{aligned}$$
(7)

subject to

$$\begin{aligned}&\sum _{r=1}^{s} \mu _{ro} y_{rj} - \sum _{i=0}^{m} \nu _{io}x_{ij}&\le 0\quad \mathrm{\ for\ } j= 1,\ldots ,n \end{aligned}$$
(8)
$$\begin{aligned}&\sum _{i=1}^{m} \nu _{io}x_{io}&= 1 \end{aligned}$$
(9)
$$\begin{aligned}&\mu _{ro},\nu _{io}&\ge 0 \end{aligned}$$
(10)

This formulation of the problem is called input-oriented, it determines how much the inputs of a DMU could be reduced by maintaining the same level of outputs. Some DMUs determine a Pareto frontier, or envelopment surface, in input-output space and are called efficient, for inefficient DMUs an efficient projection path to the envelopment surface is determined.

Despić Ozren (2013) showed that CCR can be reformulated using the same ratios of benefit attributes and of cost attributes as defined in Eq. 2, but in his reformulation the ratios of benefit attributes are put in a different relation to the ratios of the cost attributes as expressed in Eq. 3.

By introducing the following values for the weights \(\mu _{ro}\) and \(\nu _{io}\)

$$\begin{aligned} \mu _{ro} = \frac{1}{s} \times \frac{q_o}{y_{ro}},\ \ \nu _{io} = \frac{1}{m} \times \frac{1}{x_{io}} \end{aligned}$$
(11)

where \(q_o\) equals the relative quality of DMU(o) according to the procedure in Sect. 2.2, Eqs. 7, and 10 become

$$\begin{aligned} z_o&= \sum _{r=1}^{s} \mu _{ro} y_{ro} = q_o \end{aligned}$$
(12)
$$\begin{aligned} \sum _{i=1}^{m} \nu _{io}x_{io}&= \sum _{i=1}^{m} \frac{x_{io}}{x_{io}} = 1 \end{aligned}$$
(13)

From the definition of \(q_{j;o}\) and \(q_o\) we have

$$\begin{aligned} q_{j;o}\times q_o = \left( \frac{1}{m} \sum _{i=1}^m \frac{x_{io}}{x_{ij}} \right) \times \left( \frac{1}{s} \sum _{r=1}^s \frac{y_{rj}}{y_{ro}} \right) \times q_o \le 1\ \ \forall j \end{aligned}$$
(14)

as \(q_{j;o}.q_o\) equals \(q_{j;o}/(max_{j}(q_{j;o}), j=1,\ldots , n)\). So

$$\begin{aligned}&\left( \frac{1}{s} \sum _{r=1}^s \frac{y_{rj}\times q_o}{y_{ro}} \right) \le \frac{1}{\frac{1}{m} \sum _{i=1}^m \frac{x_{io}}{x_{ij}}}\end{aligned}$$
(15)
$$\begin{aligned}&=\,\frac{m}{\sum _{i=1}^m \frac{x_{io}}{x_{ij}}} \le \frac{1}{m} \sum _{i=1}^m \frac{x_{ij}}{x_{io}} \end{aligned}$$
(16)

the last step because the harmonic mean H is the reciprocal dual of the arithmetic mean A, i.e., \(1/H(1/x_1,\dots ,1/x_m)=A(x_1,\dots ,x_m)\). Then it holds

$$\begin{aligned} \sum _{r=1}^{s} \mu _{ro} y_{rj} - \sum _{i=0}^{m} \nu _{io}x_{ij} \le 0\ \forall j \end{aligned}$$

with

$$\begin{aligned} q_o - 1 \le 0\ \ j=o \end{aligned}$$
(17)

For \(q_o<1\) the parameters defined by Eq. 11 correspond with a feasible solution of DEA. In case \(q_o=1\), then \(z_o=1\), the maximum possible value for the relative efficiency, like for N(2) in the network use case above, and an optimal solution to problem 7 is found for alternative o with strictly positive weights \(\mu _{ro}\) and \(\nu _{io}\).

The choice for the weights chosen in Eq. 11 and the implicit weighting performed in Eqs. 3 and 2 are tightly related.

3 Comparison with different MADM methods

In this section we look at four other MADM methods and compare these with the method of the relative quality measure. The decision matrix for this comparison is shown in the Appendix in Table 7. In Table 8 values for Pearson’s correlation coefficients between the benefit and cost attributes are listed. Some values for the correlation coefficient are negative, i.e., high benefits correlate with low costs and vice versa, and some values are positive, i.e., high benefits correlates with high costs and vice versa.

Four methods are chosen, the MADM methods based on OWA operator, maximizing deviations, and information entropy, and the PROMETHEE II complete ranking method. The first three methods start with the construction of a matrix R from the decision matrix of a set of n alternatives, each alternative having m attributes. The entries \(r_{ij}\) of matrix R depend on the type of attribute j. For a benefit attribute j the column entries are defined by

$$\begin{aligned} r_{ij} = \frac{a_{ij}}{max_i\{a_{ij}\}} \quad i=1,2,\ldots , n \end{aligned}$$
(18)

and for a cost attribute j the column entries are

$$\begin{aligned} r_{ij} = \frac{min_i\{a_{ij}\}}{a_{ij}} \quad i=1,2,\ldots , n \end{aligned}$$
(19)

So each attribute is normalized with respect to the best value for this attribute among the alternatives, such that for any entry of matrix R holds \(r_{ij}\le 1\).

The PROMETHEE II method needs additional information related to the preferences and the priorities of the decision-maker. It requires a preference function associated to each criterion as well as weights describing their relative importance.

In the next four subsections we describe how each of these methods proceed and where we follow the notation from Xu (2015) for the first three methods and the notation from Greco et al. (2016) for the PROMETHEE II method.

3.1 MADM based on OWA operator

The method based on OWA operator Yager (1988) is used to aggregate all the attribute values \(r_{ij}\) of the normalized decision matrix R for each alternative i, and to arrive at a value for the degree of alternative i:

$$\begin{aligned} z_i ({\mathbf{\omega}}) = ( r_{i1}, r_{i2},\ldots , r_{im} ) = \sum _{j=0}^{m} \omega _j b_j \end{aligned}$$
(20)

where \(b_j\) is the jth largest of the arguments \(r_{ij}\), i.e., the arguments are arranged in descending order, and \({\mathbf{\omega}} =(\omega _j)\) is a weight vector. A possible choice for the weights \(\omega _j\) follows from

$$\begin{aligned} \omega _1=\frac{1-\alpha }{m}+\alpha , \ \omega _i=\frac{1-\alpha }{m}, \ \alpha \in [0,1] \end{aligned}$$
(21)

where m the number of attributes (Xu (2015) theorem 1.10). All the values \(z_i\) result in a complete ranking of the alternatives.

3.2 MADM based on maximizing deviations

This method assigns to attribute j of alternative i a value \(D_{ij}\) , denoting a weighted deviation between alternative i and all other alternatives with respect to this attribute:

$$\begin{aligned} D_{ij} = \sum _{k=1}^{n} |r_{ij} - r_{kj} | w_j \end{aligned}$$
(22)

Summing over all alternatives and attributes an objective function \(D({\mathbf{w}})\) is obtained from which the weights \(w_j\) are derived:

$$\begin{aligned} max D({\mathbf{w}}) = \sum _{j=1}^{m} \sum _{i=1}^{n} \sum _{k=1}^{n} |r_{ij} - r_{kj} | w_j \end{aligned}$$
(23)

The resulting weight vector, following a method from Wang (1998), is giving by:

$$\begin{aligned} w_j = \frac{ \sum _{i=1}^{n} \sum _{k=1}^{n} |r_{ij} - r_{kj} | }{ \sum _{j=1}^{m} \sum _{i=1}^{n} \sum _{k=1}^{n} |r_{ij} - r_{kj} | } \end{aligned}$$
(24)

To each alternative i a value \(z_i\) is assigned according to

$$\begin{aligned} z_i({\mathbf{w}} ) = \sum _{j=1}^{m} r_{ij} w_j \end{aligned}$$
(25)

Ranking is based on the values \(z_i\).

3.3 MADM based on information entropy

This method based on information entropy constructs from the normalized matrix R a matrix \(\dot{R}\) having entries:

$$\begin{aligned} \dot{r}_{ij} = \frac{r_{ij}}{\sum _{i=1}^{n} r_{ij}} \end{aligned}$$
(26)

For each attribute j an entropy value is derived according to:

$$\begin{aligned} E_j = - \frac{1}{ln\ n} \sum _{i=1}^{n} \dot{r}_{ij}\ ln\ \dot{r}_{ij}\quad \ j=1,\ldots ,m \end{aligned}$$
(27)

The derived weight vector \({\mathbf{w}}\) becomes:

$$\begin{aligned} w_j = - \frac{1 - E_j}{ \sum _{k=1}^{m} (1-E_k) } \end{aligned}$$
(28)

Applying Eq. 25 for each alternative results in a ranking of all the alternatives.

3.4 PROMETHEE II

The PROMETHEE II method Brans and Vincke (1985), a outranking method, provides a complete ranking of a set of n alternatives \(A = \{ A_1,\ldots , A_n \}\). Each alternative is characterized by a set of k evaluation criteria \(\{ g_1(.),\ldots , g_k(.) \}\). The amplitude of the deviation between criterion j for two altenatives \(A_h\) and \(A_i\) is defined as

$$\begin{aligned} d_j(A_h,A_i) = g_j(A_h) - g_j(A_i) . \end{aligned}$$
(29)

The decision maker is aksed for a preference function \(P_j\) for each criterion

$$\begin{aligned} P_j(A_h,A_i) = F_j[d_j(A_h,A_i)]\ \ \forall A_h, A_i \in A , \end{aligned}$$
(30)

and

$$\begin{aligned} 0 \le P_j(A_h,A_i) \le 1. \end{aligned}$$
(31)

For criterion j to be maximized, related to a benefit attribute, \(P_j\) has the following property

$$\begin{aligned} P_j(A_h,A_i) > 0 \Rightarrow P_j(A_i,A_h) = 0, \end{aligned}$$
(32)

whereas for a criterion j to be minimized, related to a cost attribute, the preference function takes the form

$$\begin{aligned} P_j(A_h,A_i) = F_j[-d_j(A_h,A_i)]\ \ \forall A_h, A_i \in A. \end{aligned}$$
(33)

Different types of preference function P(d) are used, where we will opt for a Type 2:

$$\begin{aligned} P(d) = {\left\{ \begin{array}{ll} 0 &{} d\le q \\ 1 &{} d > q \end{array}\right. } \end{aligned}$$
(34)

for all criteria. This choice allows for a fair comparison with the previous MADM methods, as it expresses the fact that the decion maker has no preference, except for a threshold value q for the deviation d.

Next, a preference index for alternative \(A_h\) with regard to alternative \(A_i\) over all criteria is defined

$$\begin{aligned} \pi (A_h,A_i) = \sum _{j=1}^{k} P_j(A_h,A_i)\ w_j, \end{aligned}$$
(35)

which expresses with which degree alternative \(A_h\) is preferred to alternative \(A_i\). For reasons of comparison we take all weights \(w_j\) equal, \(w_j=\frac{1}{k}\). Finally, two outranking flows are used, a positive outranking flow

$$\begin{aligned} \phi ^+(A_h) = \frac{1}{n-1} \sum _{x\in A}^{} \pi (A_h,x), \end{aligned}$$
(36)

and a negative outranking flow

$$\begin{aligned} \phi ^-(A_h) = \frac{1}{n-1} \sum _{x\in A}^{} \pi (x, A_h). \end{aligned}$$
(37)

The better an alternative \(A_h\) the higher \(\phi ^+(A_h)\), and the more alternative \(A_h\) is outranked by others, the higher \(\phi ^-(A_h)\). By considering the net outranking flow

$$\begin{aligned} \phi (A_h) = \phi ^+(A_h) - \phi ^-(A_h)\ \ \forall A_h, A_i \in A \end{aligned}$$
(38)

a complete ranking results for the alternatives in A.

3.5 MADM comparison results

The complete ranking of all five methods will be compared for the decision matrix in Table 7.

For the OWA method \(\alpha =0.2\) is chosen and the weights according to Eq. 21 become: \(\omega _1=0.28\), and \(\omega _i=0.08\ i=2,\ldots ,10\).

The weight vector following from the decision matrix in Table 7 for the method based on maximizing deviations becomes according to Eq. 25: \({\mathbf{w}}=\) (0.0630, 0.0605, 0.1072, 0.0655, 0.1136, 0.1145, 0.1294, 0.1233, 0.1363, 0.08641), which components add up to one.

For the method based on Information Entropy the weight vector defined in Eq. 28 takes for the decision matrix in Table 7 the values: \({\mathbf{w}}=\) (0.1179, 0.1818, 0.0624, 0.1358, 0.1025, 0.0797, 0.0873, 0.0766, 0.1194, 0.0362).

Concerning the PROMETHEE II method, the same type 2 preference funtion, Eq. 34, is applied to all criteria with q = 20% for the criteria C1(−) until C9(+) and q = 10% for criterium C10(+) for the decision matrix in Table 7. The positive and negative outranking flow which allow for a complete ranking, Eq. 38, take the values:

\(\phi ^+(A)=\) (0.2736, 0.2631, 0.2947, 0.3631, 0.2421, 0.1947, 0.4421, 0.2, 0.1842, 0.1684, 0.2210, 0.2526, 0.1947, 0.1105, 0.2052, 0.0947, 0.2526, 0.2, 0.1842, 0.0578) and \(\phi ^-(A)=\) (0.1263, 0.1894, 0.3157, 0.0684, 0.2473, 0.1842, 0.0947, 0.2157, 0.2315, 0.2, 0.1736, 0.1684, 0.2263, 0.2526, 0.3210, 0.3368, 0.1631, 0.2052, 0.2684, 0.4105).

In Table 5 the ranking by the different MADM methods for the alternatives in decision matrix of Table 7 is shown. What can be noticed is that all agree well in assigning the highest rank to alternative A7 and the lowest rank to alternative A20. All methods also agree well in assigning the next lowest alternative, A16. Here must be noticed that alternative A4 gets the highest rank by the PROMETHEE II method instead of A7, and alternative A7 becomes the second highest in rank if the preference threshold is taken to be q = 10% for all criteria.

A large discrepancy in ranking occurs for alternatives A5 and A10. This is due to the way the relative quality compares the attributes for each pair of alternatives. Alternative A7 has all entries \(Q(7,j)<1,\ j\ne 7\), and outperforms all other alternatives, thus having the maximum relative quality 1. Alternative A5 has a row maximum for entry \(Q(5,7)=8.6104\), resulting in a relative quality \(q_5=0.116\), the third lowest value, see Table 5. This value results from Eq. 6: \(Q(5,7)=(\frac{1}{4}(\frac{5}{3}+\frac{7}{1}+\frac{9}{3}+\frac{7}{2}))\). \((\frac{1}{6}(\frac{9}{1}+\frac{5}{3}+\frac{9}{9}+\frac{5}{9}+\frac{5}{9}+\frac{0.683}{0.806}))=8.6104\). What can be noticed are the large contributions of the terms \(\frac{x_{2,5}}{x_{2,7}}=\frac{C2(A5)}{C2(A7)}=\frac{7}{1}\) and \(\frac{y_{1,7}}{y_{1,5}}=\frac{C5(A7)}{C5(A2)}=\frac{9}{1}\). Cost attribute \(C2(-)\) and benefit attribute \(C5(+)\) are negatively correlated, see Table 8, and alternative A5 exihibits the opposite behaviour compared to the best alternative A7, i.e., A5 has a high value for \(C2(-)\) and a low value for \(C5(+)\), whereas A7 has a low value for \(C2(-)\) and a high value for \(C5(+)\). So it is reasonable to assign a low rank to alternative A5.

Alternative A10 also has a row maximum for the entry related to alternative A7, \(Q(10,7)=3.8286\), resulting in a relative quality \(q_{10}=0.261\), the seventh highest value, see Table 5. This value results from Eq. 6: \(Q(10, 7)=(\frac{1}{4}(\frac{3}{3}+\frac{5}{1}+\frac{7}{3}+\frac{3}{2}))\). \((\frac{1}{6}(\frac{9}{3}+\frac{5}{5}+\frac{9}{5}+\frac{5}{3}+\frac{5}{7}+\frac{0.683}{0.587}))=3.8286\). Here the contributions of the terms \(\frac{x_{2,10}}{x_{2,7}}\) and \(\frac{y_{1,7}}{y_{1,10}}\) are smaller then in the case of alternative A5. This results in a much higher ranking for alternative A10 then for alternative A5.

In Table 6 Spearman’s rank correlation coefficient for the rankings by the different methods in Table 6 is listed, showing that the rank correlation between the relative quality measure Q and the method of Information Entropy has the highest values compared to the rank correlation between the relative quality measure Q and the other methods.

Table 5 The ordinal ranking of the alternatives O(WA), M(axDev), E(ntropy), and Q, and the P(ROMETHEE)II method
Full size table
Table 6 Spearman’s rank correlation coefficient for the rankings in Table 5
Full size table

4 Conclusions

The observation that minimizing a cost attribute might have the side effect that certain benefit attributes may also decrease, which was illustrated by an example of energy saving strategies for a Wide Area Network (Sect. 2.1), was the motivation for constructing a relative quality measure. Instead of normalizing each attribute individually with the best value found among the alternatives (Sect. 3), the method presented starts with comparing the attributes of each alternative out of a set of n alternatives with those of all the other alternatives, resulting in a matrix Q of \(n \times n\) comparisons (Sect. 2.2). For this matrix Q a relative quality measure was defined leading to a ranking of the alternatives. The relative quality measure for an alternative o is defined as the reciprocal of the maximum of its row Q(o). As the comparison makes use of the arithmetic mean of benefit attribute ratios and the harmonic mean of cost attribute ratios, some special properties of matrix Q can be derived. For a set of distinct alternatives there can only be one alternative having the maximum relative quality of 1. The definition of the relative quality measure and the choice for the arithmetic and harmonic mean makes a relation with the method of data envelopment analysis (DEA) possible (Sect. 2.4). If an alternative has the maximum relative quality of 1 it is an extremely efficient unit in terms of DEA.

The method of the relative quality compares well with other MADM methods in ranking the best and the worse alternatives. Other alternatives besides the best and the worst can be ranked differently. Comparison was made with MADM methods based on OWA operator, maximizing deviations, and information entropy, and the PROMETHEE II method (Sect. 3).

The final objective of MADM is to help a decision maker to make “better” decisions. What “better” means depends, in part, on the process by which the decision is made and implemented, according to Bernard Roy (2016). Decision makers originating from different fields of expertise must be able to understand what is happening, and the method must be easy to implement. It are these requirements the method of the relative quality Q adheres to.

To strengthen the confidence that the method provides ‘good’ preference information, a relation with other methods will help. When do other methods agree and when do they differ and why? Different problems having decision matrices with special features, like how cost and benefit attributes are correlated, might help to answer this question. The choice for suitable MADM methods by a decision maker might then be based upon the essential features of the decision matrix at hand.