Rank ordering engineering designs: pairwise comparison charts and Borda counts

Abstract.

Designers routinely rank alternatives in a variety of settings using a staple of comparison, the pairwise comparison. In recent years questions have been raised about the use of such comparisons as a means of calculating and aggregating meaningful preference or choice data. Results on voting have been used to argue that the positional procedure known as the Borda count is the best pairwise voting procedure, or at least the only one that is not subject to a number of demonstrable problems. We show here that pairwise comparison charts (PCC) provide results that are identical to those obtained by the Borda count, and that the PCC is thus not subject to the arguments used against non-Borda count methods. Arrow's impossibility theorem has also been invoked to cast doubt upon any pairwise procedure, including the Borda count. We discuss the relevance of the Arrow property that is lost in the Borda count, the independence of irrelevant alternatives (IIA). While the theoretical consequences of IIA are devastating, it is not clear that the same is true of its practical consequences. Examples are presented to illustrate some primary objections to pairwise methods.