Annals of Operations Research

, Volume 211, Issue 1, pp 511–528 | Cite as

Analysis of pairwise comparison matrices: an empirical research

  • Sándor Bozóki
  • Linda Dezső
  • Attila Poesz
  • József TemesiEmail author


Pairwise comparison (PC) matrices are used in multi-attribute decision problems (MADM) in order to express the preferences of the decision maker. Our research focused on testing various characteristics of PC matrices. In a controlled experiment with university students (N=227) we have obtained 454 PC matrices. The cases have been divided into 18 subgroups according to the key factors to be analyzed. Our team conducted experiments with matrices of different size given from different types of MADM problems. Additionally, the matrix elements have been obtained by different questioning procedures differing in the order of the questions. Results are organized to answer five research questions. Three of them are directly connected to the inconsistency of a PC matrix. Various types of inconsistency indices have been applied. We have found that the type of the problem and the size of the matrix had impact on the inconsistency of the PC matrix. However, we have not found any impact of the questioning order. Incomplete PC matrices played an important role in our research. The decision makers behavioral consistency was as well analyzed in case of incomplete matrices using indicators measuring the deviation from the final order of alternatives and from the final score vector.


Multi-attribute decision making Experimental techniques in decision making Pairwise comparisons Inconsistency Incomplete pairwise comparison matrix 


  1. Aguarón, J., & Moreno-Jiménez, J. M. (2003). The geometric consistency index: approximated thresholds. European Journal of Operational Research, 147, 137–145. CrossRefGoogle Scholar
  2. Bana e Costa, C. A., & Vansnick, J.-C. (2008). A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research, 187, 1422–1428. CrossRefGoogle Scholar
  3. Bozóki, S., Fülöp, J., & Koczkodaj, W. W. (2011a). An LP-based inconsistency monitoring of pairwise comparison matrices. Mathematical and Computer Modelling, 54(1–2), 789–793. CrossRefGoogle Scholar
  4. Bozóki, S., Fülöp, J., & Poesz, A. (2011b). On pairwise comparison matrices that can be made consistent by the modification of a few elements. Central European Journal of Operations Research, 19(2), 157–175. CrossRefGoogle Scholar
  5. Bozóki, S., Fülöp, J., & Rónyai, L. (2010). On optimal completions of incomplete pairwise comparison matrices. Mathematical and Computer Modelling, 52, 318–333. CrossRefGoogle Scholar
  6. Bozóki, S., & Rapcsák, T. (2008). On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. Journal of Global Optimization, 42(2), 157–175. CrossRefGoogle Scholar
  7. Brunelli, M., & Fedrizzi, M. (2011). Characterizing properties for inconsistency indices in the AHP. In Proceedings of the international symposium on the analytic hierarchy process (ISAHP), 15–18 Sorrento (Naples), Italy. Google Scholar
  8. Choo, E. U., & Wedley, W. C. (2004). A common framework for deriving preference values from pairwise comparison matrices. Computers & Operations Research, 31, 893–908. CrossRefGoogle Scholar
  9. Gass, S. (1998). Tournaments, transitivity and pairwise comparison matrices. Journal of the Operational Research Society, 49, 616–624. Google Scholar
  10. Gass, S. I., & Rapcsák, T. (2004). Singular value decomposition in AHP. European Journal of Operational Research, 154, 573–584. CrossRefGoogle Scholar
  11. Gass, S. I., & Standard, S. M. (2002). Characteristics of positive reciprocal matrices in the analytic hierarchy process. Journal of the Operational Research Society, 53(12), 1385–1389. CrossRefGoogle Scholar
  12. Harker, P. T. (1987). Incomplete pairwise comparisons in the analytic hierarchy process. Mathematical Modelling, 9(11), 837–848. CrossRefGoogle Scholar
  13. Kendall, M. G., & Smith, B. B. (1940). On the method of paired comparisons. Biometrika, 31(3–4), 324–345. Google Scholar
  14. Kéri, G. (2011). On qualitatively consistent, transitive and contradictory judgment matrices emerging from multiattribute decision procedures. Central European Journal of Operations Research, 19, 215–224. CrossRefGoogle Scholar
  15. Koczkodaj, W. W. (1993). A new definition of inconsistency of pairwise comparisons. Mathematical and Computer Modelling, 8, 79–84. CrossRefGoogle Scholar
  16. Koczkodaj, W. W., Herman, M. W., & Orlowski, M. (1997). Using consistency-driven pairwise comparisons in knowledge-based systems. In Proceedings of the sixth international conference on information and knowledge management (pp. 91–96). New York: ACM. Google Scholar
  17. Linares, P. (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. European Journal of Operational Research, 193, 492–498. CrossRefGoogle Scholar
  18. Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81–97. CrossRefGoogle Scholar
  19. Murphy, C. K. (1993). Limits on the analytic hierarchy process from its inconsistency index. European Journal of Operational Research, 65, 138–139. CrossRefGoogle Scholar
  20. Peláez, J. I., & Lamata, M. T. (2003). A new measure of consistency for positive reciprocal matrices. Computers & Mathematics With Applications, 46, 1839–1845. CrossRefGoogle Scholar
  21. Poesz, A. (2008). Inconsistency analysis of empirical pairwise comparison matrices. Master’s thesis, Corvinus University of Budapest (in Hungarian). Google Scholar
  22. Ross, R. T. (1934). Optimum orders for the presentation of pairs in the method of paired comparison. Journal of Educational Psychology, 25, 375–382. CrossRefGoogle Scholar
  23. Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill. Google Scholar
  24. Saaty, T. L., & Ozdemir, M. S. (2003). Why the magic number seven plus or minus two. Mathematical and Computer Modelling, 38(3–4), 233–244. doi: 10.1016/S0895-7177(03)90083-5. CrossRefGoogle Scholar
  25. Saaty, T. L. (2005). The analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple criteria decision analysis: state of the art surveys (pp. 345–407). New York: Springer. Google Scholar
  26. Temesi, J. (2006). Consistency of the decision-maker in pairwise comparisons. International Journal of Management and Decision Making, 7(2–3), 267–274. CrossRefGoogle Scholar
  27. Temesi, J. (2011). Pairwise comparison matrices and the error-free property of the decision maker. Central European Journal of Operations Research, 19(2), 239–249. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Sándor Bozóki
    • 1
    • 2
  • Linda Dezső
    • 3
  • Attila Poesz
    • 2
  • József Temesi
    • 2
    Email author
  1. 1.Computer and Automation Research InstituteHungarian Academy of Sciences (MTA SZTAKI)BudapestHungary
  2. 2.Department of Operations Research and Actuarial SciencesCorvinus University of BudapestBudapestHungary
  3. 3.Department of Applied Psychology: Work, Education and EconomyUniversity of ViennaViennaAustria

Personalised recommendations