A generalization of the Thurstone method for multiple choice and incomplete paired comparisons

  • Éva Orbán-Mihálykó
  • Csaba Mihálykó
  • László Koltay
Original Paper


A ranking method based on paired comparisons is proposed. The object’s characteristics are considered as random variables and the observers judge about their differences. The differences are classified. More than two classes are allowed. Assuming Gauss distributed latent random variables we set up the likelihood function and estimate the parameters by the maximum likelihood method. The rank of the objects is the order of the expectations. We analyse the log-likelihood function and provide reasonable conditions for the existence of the maximum value and the uniqueness of the maximizer. Some illustrative examples are also presented. The method can be applied in case of incomplete comparisons as well. It allows constructing confidence intervals for the probabilities and testing the hypothesis that there are no significant differences between the expectations.


Paired comparison Thurstone’s method Multiple options Maximum likelihood estimation Likelihood ratio test Incomplete comparison 


  1. Agresti A (1992) Analysis of ordinal paired comparison data. Appl Stat 41(2):287–297CrossRefGoogle Scholar
  2. Basak I (2015) A statistical hypothesis testing method for the rank ordering of the priorities of the alternatives in the analytic hierarchy process. J Multi-Criteria Decis Anal 22(3–4):161–166CrossRefGoogle Scholar
  3. Baker RD, McHale IG (2014) A dynamic paired comparisons model: who is the greatest tennis player? Eur J Oper Res 236(2):677–684CrossRefGoogle Scholar
  4. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  5. Bozóki S, Fülöp J, Rónyai L (2010) On optimal completion of incomplete pairwise comparison matrices. Math Comput Model 52(1):318–333CrossRefGoogle Scholar
  6. Bozóki S, Csató L, Temesi J (2016) An application of incomplete pairwise comparison matrices for ranking top tennis players. Eur J Oper Res 248(1):211–218CrossRefGoogle Scholar
  7. Bradley RA, Terry ME (1952) Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3/4):324–345CrossRefGoogle Scholar
  8. Cattelan M (2012) Models for paired comparison data: a review with emphasis on dependent data. Stat Sci 27(3):412–433CrossRefGoogle Scholar
  9. Csató L (2017) On the ranking of a Swiss system chess team tournament. Ann Oper Res 254(1–2):17–36CrossRefGoogle Scholar
  10. Franceschini F, Maisano D, Mastrogiacomo L (2015) A paired-comparison approach for fusing preference orderings from rank-ordered agents. Inf Fusion 26:84–95CrossRefGoogle Scholar
  11. Mosteller F (1951) Remarks on the method of paired comparisons: I. The least squares solution assuming equal standard deviations and equal correlations. Psychometrika 16(1):3–9CrossRefGoogle Scholar
  12. Orbán-Mihálykó É, Koltay L, Szabó F, Csuti P, Kéri R, Schanda J (2015) A new statistical method for ranking of light sources based on subjective points of view. Acta Polytech Hung 12(8):195–214Google Scholar
  13. Pfeiffer T, Gao XA, Chen Y, Mao A, Rand D G (2012) Adaptive polling for information aggregation. In: AAAIGoogle Scholar
  14. Prékopa A (1973) Logarithmic concave measures and functions. Acta Sci Math 34(1):334–343Google Scholar
  15. Rao PV, Kupper LL (1967) Ties in paired-comparison experiments: a generalization of the Bradley–Terry model. J Am Stat Assoc 62(317):194–204CrossRefGoogle Scholar
  16. Saaty TL (1990) Multicriteria decision making: the analytic hierarchy Process: planning, priority setting resource allocation. RWS Publications, PittsburghGoogle Scholar
  17. Saaty TL, Hu G (1998) Ranking by eigenvector versus other methods in the analytic hierarchy process. Appl Math Lett 11(4):121–125CrossRefGoogle Scholar
  18. Stern H (1990) A continuum of paired comparisons models. Biometrika 77(2):265–273CrossRefGoogle Scholar
  19. Szabó F, Kéri R, Schanda J, Csuti P, Mihálykó-Orbán É (2016a) A study of preferred colour rendering of light sources: home lighting. Light Res Technol 48(2):103–125CrossRefGoogle Scholar
  20. Szabó F, Kéri R, Schanda J, Csuti P, Wilm A, Baur E (2016b) A study of preferred colour rendering of light sources: shop lighting. Light Res Technol 48(3):286–306CrossRefGoogle Scholar
  21. Tutz G (1986) Bradley–Terry–Luce models with an ordered response. J Math Psychol 30(3):306–316CrossRefGoogle Scholar
  22. Thurstone LL (1927) A law of comparative judgment. Psychol Rev 34(4):273–286CrossRefGoogle Scholar
  23. Ureńa R, Chiclana F, Morente-Molinera JA, Herrera-Viedma E (2015) Managing incomplete preference relations in decision making: a review and future trends. Inf Sci 302:14–32CrossRefGoogle Scholar
  24. Vaidya OS, Kumar S (2006) Analytic hierarchy process: an overview of applications. Eur J Oper Res 169(1):1–29CrossRefGoogle Scholar
  25. Wilks SS (1938) The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann Math Stat 9(1):60–62CrossRefGoogle Scholar
  26. Yurtcu M, Dogan N (2015) Scaling with paired comparison method for reasons for mathematics anxiety of secondary school students. Procedia Soc Behav Sci 197:851–857CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of PannoniaVeszprémHungary

Personalised recommendations