Ranking Aggregation Techniques

Abstract

This chapter focuses on ranking aggregation techniques, which are the core of the whole book. The initial part of the chapter suggests a taxonomy based on three characteristic features: (a) input data characteristics, (b) aggregation mechanism, and (c) output data characteristics.

Without any ambition of exhaustiveness, the rest of the chapter offers a varied description of state-of-the-art techniques, ranging from some very popular and well-established ones—such as those from Voting Theory (Borda’s Count, Instant-Runoff Voting, etc.), ELECTRE-II method, Yager’s algorithm (YA), and Thurstone’s Law of Comparative Judgment (LCJ)—to more recent and innovative ones—such as (1) Enhanced Yager’s algorithm (EYA) and (2) ZM_II technique. A structured case study application accompanies the description.

Appendix

5.1.1 Further Example

This section integrates Sect. 5.3, showing an additional case study example about the application of five Voting Theory’s aggregation techniques: BoB, BTW, BTH, IRV, and BC.

A hi-tech company, which operates predominantly in the video-projector industry, wants to develop a new model of hand-held projector, also known as a pocket projector, mobile projector, pico-projector, or mini beamer. Four design concepts of pocket projectors (i.e., the objects of the problem: o₁ to o₄) have been generated by a team of ten engineering designers (i.e., the experts of the problem: e₁ to e₁₀), during the conceptual design phase (Franceschini & Maisano, 2021a):

(o₁):

Stand-alone projector

(o₂):

USB projector

(o₃):

Media player projector

(o₄):

Embedded-type projector

The objective is to evaluate the aforementioned design concepts in terms of user-friendliness , i.e., a measure of the ease of use of a pocket projector. Some of the factors that can positively influence this attribute are: (1) quick set-up time, (2) intuitive controls, and (3) good user interface.

Given the great difficulty in bringing together all the experts and making them interact to reach shared decisions, management leaned toward a different solution: a collective ranking of the four design concepts can be obtained by merging the individual rankings formulated by the ten engineering designers (in Table 5.37).

Inspired by different design strategies, the team of engineering designers decides to consider five popular aggregation techniques from the scientific literature: BoB, BTW, BTH, IRV, and BC, illustrated in Sects. 5.3.1 to 5.3.4 (Saari, 2011). Table 5.38 contains the results obtained when applying these techniques.

Table 5.37 (Strict linear) rankings of four design concepts (i.e., o₁ to o₄) formulated by ten engineering designers (i.e., e₁ to e₁₀)

While the application of BoB, BTW, and BTH techniques is relatively trivial and therefore left to the reader, the application of the IRV technique deserves more attention. First, it can be noticed that object o₂ never appears as the first choice in the initial expert rankings (in Table 5.37); this object can therefore be eliminated in advance, as it can never compete for victory. In the first round, the design concept o₁ thus obtains five first choices, o₃ obtains two first choices, and o₄ obtains three first choices; since no object has obtained more than half of the preferences based on first choices, o₃—i.e., the one with fewest first choices—is eliminated. In the second round—which represents the so-called “instant runoff,” in this case—a tie is observed between o₁ and o₄, both obtaining five first choices.

As for the BC technique (illustrated in Sect. 5.3.4), the cumulative scores—or Borda counts, BC(o_i)—of the four design concepts are calculated as:

$$ {\displaystyle \begin{array}{l}\mathrm{BC}\left({o}_1\right)=1+1+1+1+1+4+4+4+4+4=25\\ {}\mathrm{BC}\left({o}_2\right)=2+2+4+4+4+2+2+2+2+2=26\\ {}\mathrm{BC}\left({o}_3\right)=3+3+2+3+3+1+1+3+3+3=25\\ {}\mathrm{BC}\left({o}_4\right)=4+4+3+2+2+3+3+1+1+1=24\end{array}} $$

(5.41)

Reflecting different design strategies, the five aggregation techniques produce five different collective rankings (see the last column of Table 5.38). Even more surprising is that the best pocket projector design concept (i.e., the object at the top of each collective ranking) is (almost) different for each of the five aggregation models!

Although this plurality of results may at first glance confuse the reader, it is in some measure justified by the low concordance of the expert rankings. Considering that the problem of interest does not include any ranking with ties, the formula in Eq. (4.11) (Sect. 4.4) can be used to determine the Kendall’s concordance coefficient W^(m) = 0.004 = 0.4%, which denotes a very low degree of concordance among experts. In view of this remarkable dispersion in the expert rankings, it is not surprising to observe discrepancies between the techniques in designating the winning objects.

Table 5.38 Scores related to the design concepts (i.e., o₁ to o₄) and corresponding collective ranking, produced by each of the four aggregation techniques (i.e., BoB, BTW, IRV, and BC)