Skip to main content
SpringerLink
Log in

Analysis of pairwise comparison matrices: an empirical research

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Pairwise comparison (PC) matrices are used in multi-attribute decision problems (MADM) in order to express the preferences of the decision maker. Our research focused on testing various characteristics of PC matrices. In a controlled experiment with university students (N=227) we have obtained 454 PC matrices. The cases have been divided into 18 subgroups according to the key factors to be analyzed. Our team conducted experiments with matrices of different size given from different types of MADM problems. Additionally, the matrix elements have been obtained by different questioning procedures differing in the order of the questions. Results are organized to answer five research questions. Three of them are directly connected to the inconsistency of a PC matrix. Various types of inconsistency indices have been applied. We have found that the type of the problem and the size of the matrix had impact on the inconsistency of the PC matrix. However, we have not found any impact of the questioning order. Incomplete PC matrices played an important role in our research. The decision makers behavioral consistency was as well analyzed in case of incomplete matrices using indicators measuring the deviation from the final order of alternatives and from the final score vector.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Aguarón, J., & Moreno-Jiménez, J. M. (2003). The geometric consistency index: approximated thresholds. European Journal of Operational Research, 147, 137–145.

    Article  Google Scholar 

  • Bana e Costa, C. A., & Vansnick, J.-C. (2008). A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research, 187, 1422–1428.

    Article  Google Scholar 

  • Bozóki, S., Fülöp, J., & Koczkodaj, W. W. (2011a). An LP-based inconsistency monitoring of pairwise comparison matrices. Mathematical and Computer Modelling, 54(1–2), 789–793.

    Article  Google Scholar 

  • Bozóki, S., Fülöp, J., & Poesz, A. (2011b). On pairwise comparison matrices that can be made consistent by the modification of a few elements. Central European Journal of Operations Research, 19(2), 157–175.

    Article  Google Scholar 

  • Bozóki, S., Fülöp, J., & Rónyai, L. (2010). On optimal completions of incomplete pairwise comparison matrices. Mathematical and Computer Modelling, 52, 318–333.

    Article  Google Scholar 

  • Bozóki, S., & Rapcsák, T. (2008). On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. Journal of Global Optimization, 42(2), 157–175.

    Article  Google Scholar 

  • Brunelli, M., & Fedrizzi, M. (2011). Characterizing properties for inconsistency indices in the AHP. In Proceedings of the international symposium on the analytic hierarchy process (ISAHP), 15–18 Sorrento (Naples), Italy.

    Google Scholar 

  • Choo, E. U., & Wedley, W. C. (2004). A common framework for deriving preference values from pairwise comparison matrices. Computers & Operations Research, 31, 893–908.

    Article  Google Scholar 

  • Gass, S. (1998). Tournaments, transitivity and pairwise comparison matrices. Journal of the Operational Research Society, 49, 616–624.

    Google Scholar 

  • Gass, S. I., & Rapcsák, T. (2004). Singular value decomposition in AHP. European Journal of Operational Research, 154, 573–584.

    Article  Google Scholar 

  • Gass, S. I., & Standard, S. M. (2002). Characteristics of positive reciprocal matrices in the analytic hierarchy process. Journal of the Operational Research Society, 53(12), 1385–1389.

    Article  Google Scholar 

  • Harker, P. T. (1987). Incomplete pairwise comparisons in the analytic hierarchy process. Mathematical Modelling, 9(11), 837–848.

    Article  Google Scholar 

  • Kendall, M. G., & Smith, B. B. (1940). On the method of paired comparisons. Biometrika, 31(3–4), 324–345.

    Google Scholar 

  • Kéri, G. (2011). On qualitatively consistent, transitive and contradictory judgment matrices emerging from multiattribute decision procedures. Central European Journal of Operations Research, 19, 215–224.

    Article  Google Scholar 

  • Koczkodaj, W. W. (1993). A new definition of inconsistency of pairwise comparisons. Mathematical and Computer Modelling, 8, 79–84.

    Article  Google Scholar 

  • Koczkodaj, W. W., Herman, M. W., & Orlowski, M. (1997). Using consistency-driven pairwise comparisons in knowledge-based systems. In Proceedings of the sixth international conference on information and knowledge management (pp. 91–96). New York: ACM.

    Google Scholar 

  • Linares, P. (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. European Journal of Operational Research, 193, 492–498.

    Article  Google Scholar 

  • Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81–97.

    Article  Google Scholar 

  • Murphy, C. K. (1993). Limits on the analytic hierarchy process from its inconsistency index. European Journal of Operational Research, 65, 138–139.

    Article  Google Scholar 

  • Peláez, J. I., & Lamata, M. T. (2003). A new measure of consistency for positive reciprocal matrices. Computers & Mathematics With Applications, 46, 1839–1845.

    Article  Google Scholar 

  • Poesz, A. (2008). Inconsistency analysis of empirical pairwise comparison matrices. Master’s thesis, Corvinus University of Budapest (in Hungarian).

  • Ross, R. T. (1934). Optimum orders for the presentation of pairs in the method of paired comparison. Journal of Educational Psychology, 25, 375–382.

    Article  Google Scholar 

  • Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill.

    Google Scholar 

  • Saaty, T. L., & Ozdemir, M. S. (2003). Why the magic number seven plus or minus two. Mathematical and Computer Modelling, 38(3–4), 233–244. doi:10.1016/S0895-7177(03)90083-5.

    Article  Google Scholar 

  • Saaty, T. L. (2005). The analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple criteria decision analysis: state of the art surveys (pp. 345–407). New York: Springer.

    Google Scholar 

  • Temesi, J. (2006). Consistency of the decision-maker in pairwise comparisons. International Journal of Management and Decision Making, 7(2–3), 267–274.

    Article  Google Scholar 

  • Temesi, J. (2011). Pairwise comparison matrices and the error-free property of the decision maker. Central European Journal of Operations Research, 19(2), 239–249.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI), Budapest, Hungary

    Sándor Bozóki

  2. Department of Operations Research and Actuarial Sciences, Corvinus University of Budapest, Budapest, Hungary

    Sándor Bozóki, Attila Poesz & József Temesi

  3. Department of Applied Psychology: Work, Education and Economy, University of Vienna, Vienna, Austria

    Linda Dezső

Authors
  1. Sándor Bozóki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linda Dezső
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Attila Poesz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. József Temesi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to József Temesi.

Additional information

The research was supported by the Hungarian Scientific Research Fund, Grant K 77420.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bozóki, S., Dezső, L., Poesz, A. et al. Analysis of pairwise comparison matrices: an empirical research. Ann Oper Res 211, 511–528 (2013). https://doi.org/10.1007/s10479-013-1328-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-013-1328-1

Keywords

Search

Navigation