Social Choice and Welfare

, Volume 42, Issue 1, pp 139–169 | Cite as

Paired comparisons analysis: an axiomatic approach to ranking methods

  • Julio González-DíazEmail author
  • Ruud Hendrickx
  • Edwin Lohmann
Original Paper


In this paper we present an axiomatic analysis of several ranking methods for general tournaments. We find that the ranking method obtained by applying maximum likelihood to the (Zermelo-)Bradley-Terry model, the most common method in statistics and psychology, is one of the ranking methods that perform best with respect to the set of properties under consideration. A less known ranking method, generalised row sum, performs well too. We also study, among others, the fair bets ranking method, widely studied in social choice, and the least squares method.


Social Choice Ranking Method Ranking Problem Rating Vector Recursive Performance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank Miguel Brozos-Vázquez and José Carlos Díaz-Ramos for helpful discussions. We also thank Pavel Chebotarev and an anonymous referee for their comments on earlier versions of the manuscript. Julio González Díaz acknowledges the financial support of the Spanish Ministry for Science and Innovation through project MTM2011-27731-c03, and from the Xunta de Galicia through project INCITE09-207-064-PR.


  1. Arrow KJ (1963) Social choice and individual values. Wiley, New YorkGoogle Scholar
  2. Berman A, Plemmons R (1994) Nonnegative matrices in the mathematical sciences, 2nd edn. SIAM, PhiladelphiaCrossRefGoogle Scholar
  3. Birkhoff G (1946) Tres observaciones sobre el álgebra lineal. Univ Nac Tucumán Revista A 5:147–150Google Scholar
  4. Borm P, Fiestras-Janeiro G, Hamers H, Sánchez-Rodríguez E, Voorneveld M (2002) On the convexity of games corresponding with sequencing situations with due dates. Eur J Oper Res 136:616–634CrossRefGoogle Scholar
  5. Bouyssou D (2004) Monotonicity of ranking by choosing: a progress report. Soc Choice Welf 23:249–273CrossRefGoogle Scholar
  6. Bradley R, Terry M (1952) Rank analysis of incomplete block designs. I. The method of paired comparisons. Biometrika 39:324–345Google Scholar
  7. Brozos-Vázquez M, Campo-Cabana MA, Díaz-Ramos JC, González-Díaz J (2008) Ranking participants in tournaments by means of rating functions. J Math Econ 44:1246–1256CrossRefGoogle Scholar
  8. Chebotarev P (1989) Generalization of the row sum method for incomplete paired comparisons. Autom Remote Control 50:1103–1113Google Scholar
  9. Chebotarev P (1994) Aggregation of preferences by the generalized row sum method. Mathe Soc Sci 27:293–320CrossRefGoogle Scholar
  10. Chebotarev P, Shamis E (1997) Econometric Decision Models: Constructing Scalar-Valued Objective Functions, of Lecture Notes in Economics and Mathematical Systems. In: Tangian A, Gruber J (eds) Constructing an objective function for aggregating incomplete preferences, vol 453. Springer-Verlag, Berlin, pp 100–124Google Scholar
  11. Chebotarev P, Shamis E (1998) Characterizations of scoring methods for preference aggregation. Ann Oper Res 80:299–332CrossRefGoogle Scholar
  12. Chebotarev P, Shamis E (1999) Preference fusion when the number of alternatives exceeds two: indirect scoring procedures. J Frankl Inst 336:205–226CrossRefGoogle Scholar
  13. Conner GR, Grant CP (2009) Neighborhood monotonicity, the extended Zermelo model, and symmetric knockout tournaments. Discret Math 309:3998–4010CrossRefGoogle Scholar
  14. Copeland AH (1951) A reasonable social welfare function. In: Seminar on Applications of Mathematics to the Social sciences. University of MichiganGoogle Scholar
  15. Daniels H (1969) Round-robin tournament scores. Biometrika 56:295–299CrossRefGoogle Scholar
  16. David HA (1988) The method of paired comparisons. Chapman and Hall, LondonGoogle Scholar
  17. Dutta B, Laslier J (1999) Comparison functions and choice correspondences. Soc Choice Welf 16:513–552CrossRefGoogle Scholar
  18. Dykstra O (1956) A note on the rank analysis of incomplete block designs: applications beyond the scope of existing tables. Biometrics 3:301–306CrossRefGoogle Scholar
  19. Ford LR (1957) Solution of a ranking problem from binary comparisons. Am Math Mon 64:28–33CrossRefGoogle Scholar
  20. Gulliksen H (1956) A least squares solution for paired comparisons with incomplete data. Psychometrika 21:125–134CrossRefGoogle Scholar
  21. Herings P, van der Laan G, Talman D (2005) The positional power nodes in digraphs. Soc Choice Welf 24:439–454CrossRefGoogle Scholar
  22. Horst P (1932) A method for determining the absolute affective value of a series of stimulus situations. J Educational Psychol 23:418–440CrossRefGoogle Scholar
  23. Kaiser HF, Serlin RC (1978) Contributions to the method of paired comparisons. Appl Psychol Meas 2:421–430CrossRefGoogle Scholar
  24. Kendall M (1955) Further contributions to the theory of paired comparisons. Biometrics 11:43–62CrossRefGoogle Scholar
  25. Langville AN, Meyer CD (2012) The science of rating and ranking. Princeton University Press, PrincetonGoogle Scholar
  26. Laslier J (1992) Solutions de tournois. Technical report, Habilitation à diriger les recherches en Economie, Université de Cergy-PontoiseGoogle Scholar
  27. Laslier J (1997) Tournament solutions and majority voting. Springer-Verlag, New YorkCrossRefGoogle Scholar
  28. Levchenkov V (1992) Social choice theory: a new insight. Technical report, Institute of Systems Analysis, MoscowGoogle Scholar
  29. Liebowitz S, Palmer J (1984) Assesing the relative impacts of economic journals. J Econ Lit 22:77–88Google Scholar
  30. Moon JW, Pullman N (1970) On generalized tournament matrices. SIAM Rev 12:384–399CrossRefGoogle Scholar
  31. Mosteller F (1951) Remarks on the method of paired comparisons. I. The least squares solution assuming equal standard deviations and equal correlations. Psychometrika 16:3–11CrossRefGoogle Scholar
  32. Neustadtl H (1882) Letter to The Chess Monhtly, London. In: L. Hoffer and J. Zukertort edsGoogle Scholar
  33. Palacios-Huerta I, Volij O (2004) The measurement of intellectual influence. Econometrica 72:963–977CrossRefGoogle Scholar
  34. Rubinstein A (1980) Ranking the participants in a tournament. SIAM J Appl Math 38:108–111CrossRefGoogle Scholar
  35. Slikker M, Borm P, van den Brink R (2010) Internal slackening scoring methods. Beta working paper, WP-306. Technische Universiteit Eindhoven, EindhovenGoogle Scholar
  36. Slutzki G, Volij O (2005) Ranking participants in generalized tournaments. Int J Game Theory 33:255–270CrossRefGoogle Scholar
  37. Slutzki G, Volij O (2006) Scoring of web pages and tournaments: axiomatizations. Soc Choice Welf 26: 75–92CrossRefGoogle Scholar
  38. Wei T (1952) The algebraic foundations of ranking theory. Ph. D. thesis, Cambridge University.Google Scholar
  39. World Chess Federation (2012) Handbook. C: General rules and recommendations for tournaments. Annex to the FIDE tournament regulations regarding tiebreaks.
  40. Zermelo E (1929) Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung. Mathematische Zeitschrift 29:436–460CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Julio González-Díaz
    • 1
    Email author
  • Ruud Hendrickx
    • 2
  • Edwin Lohmann
    • 3
  1. 1.Department of Statistics and Operations ResearchUniversity of Santiago de CompostelaSantiago de CompostelaSpain
  2. 2.CentER and Department of Organization and StrategyTilburg UniversityTilburgNetherlands
  3. 3.CentER and Department of Econometrics and Operations Research Tilburg UniversityTilburgNetherlands

Personalised recommendations