Skip to main content
SpringerLink
Log in

Specialists’ knowledge and cognitive stress in making pairwise comparisons

  • Application Article
  • Published:
OPSEARCH Aims and scope Submit manuscript

Abstract

There is no lack of studies dealing with the consistency of evaluations performed by pairwise comparison in the decision-making literature. Mostly, these studies offer algorithms for reducing the inconsistency of evaluations and indices to measure the evaluation’s consistency degree. The focus on these two research fronts does not cover all the gaps associated with the inconsistent evaluation problem. The existing algorithms are difficult to implement and do not preserve the original evaluations since the original evaluation matrix is replaced with a new matrix. Furthermore, the inconsistency of pairwise comparison has been associated with the specialist’s bounded rationality only at the theoretical-conceptual level. This research investigates the relationship between the lack of specialist knowledge and the inconsistency of evaluations, as well as introduces an approach that ensures the evaluation’s consistency by reducing the specialist’s cognitive stress when comparing a high number of alternatives. The results reveal that the specialist’s limited knowledge about the topic does not impact the degree of consistency of the evaluations as expected. The evaluation’s consistency degree is 59% lower when the specialist does have no knowledge about the decision topic but has theoretical knowledge and experience in evaluating alternatives by pairwise comparison. This is a remarkable contribution with a high degree of universality and applicability because instructing decision-makers on the inconsistency problem is a cheaper, easier way to increase the evaluation’s consistency degree without altering the original information. Furthermore, the introduced approach reduces the number of evaluations and evaluation time by 8.0 and 7.8 times, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

Silva, LML, Libório, MP (2022) A new Ease of Doing Business Index for G20 countries: a consensus-based approach to weighting individual indicators. Mendeley Data, V2, doi: https://doi.org/10.17632/kwss6jyfxk.2.

Notes

  1. Koczkodaj’s [31] Consistency Measure is calculated from a given element of the matrix and not on the characteristics of the global matrix as an eigenvalue. Salo and Hämäläinen’s [50] Consistency Measure is invariant to the scale and is calculated after transforming inconsistent responses into a non-empty set of viable priorities.

References

  1. Amenta, P., Lucadamo, A., Marcarelli, G.: On the transitivity and consistency approximated thresholds of some consistency indices for pairwise comparison matrices. Inf. Sci. 507, 274–287 (2020)

    Article  Google Scholar 

  2. Bampatsou, C., Halkos, G., Astara, O.H.: Composite indicators in evaluating tourism performance and seasonality. Oper. Res. Int. J., 1–24 (2020)

  3. Becker, W., Saisana, M., Paruolo, P., Vandecasteele, I.: Weights and importance in composite indicators: closing the gap. Ecol. Ind. 80, 12–22 (2017)

    Article  Google Scholar 

  4. Benítez, J., Delgado-Galván, X., Izquierdo, J., Pérez-García, R.: Achieving matrix consistency in AHP through linearization. Appl. Math. Model. 35(9), 4449–4457 (2011)

    Article  MathSciNet  Google Scholar 

  5. Benítez, J., Delgado-Galván, X., Izquierdo, J., Pérez-García, R.: Improving consistency in AHP decision-making processes. Appl. Math. Comput. 219(5), 2432–2441 (2012)

    Article  MathSciNet  Google Scholar 

  6. Bernardes, P., Ekel, P.I., Rezende, S.F.L., Pereira Júnior, J.G., dos Santos, A.C.G., da Costa, M.A.R., Libório, M.P.: Cost of doing business index in Latin America. Qual. Quant. 56(4), 2233–2252 (2022)

    Article  Google Scholar 

  7. Boggia, A., Fagioli, F.F., Paolotti, L., Ruiz, F., Cabello, J.M., Rocchi, L.: Using accounting dataset for agricultural sustainability assessment through a multi-criteria approach: an Italian case study. Int. Trans. Oper. Res. (2022)

  8. Bortot, S., Brunelli, M., Fedrizzi, M., Pereira, R.M.: A novel perspective on the inconsistency indices of reciprocal relations and pairwise comparison matrices. Fuzzy Sets Syst. (2022)

  9. Bozóki, S., Fülöp, J., Poesz, A.: On reducing inconsistency of pairwise comparison matrices below an acceptance threshold. Cent. Eur. J. Oper. Res. 23(4), 849–866 (2015)

    Article  MathSciNet  Google Scholar 

  10. Bustince, H., Bedregal, B., Campión, M.J., Silva, D.I., Fernandez, J., Induráin, E., Santiago, R.H.: Aggregation of individual rankings through fusion functions: criticism and optimality analysis. IEEE Trans. Fuzzy Syst. 30(3), 638–648 (2020)

    Article  Google Scholar 

  11. Camanho, A.S., Stumbriene, D., Barbosa, F., Jakaitiene, A.: The assessment of performance trends and convergence in education and training systems of European countries. Eur. J. Oper. Res. (2022)

  12. Chen, X., Zhang, H., Dong, Y.: The fusion process with heterogeneous preference structures in group decision making: a survey. Inform. Fusion 24, 72–83 (2015)

    Article  Google Scholar 

  13. Cherchye, L., Moesen, W., Rogge, N., Van Puyenbroeck, T., Saisana, M., Saltelli, A., Tarantola, S.: Creating composite indicators with DEA and robustness analysis: the case of the technology achievement index. J. Oper. Res. Soc. 59(2), 239–251 (2008)

    Article  Google Scholar 

  14. Chiclana, F., Herrera, F., Herrera-Viedma, E.: Integrating three representation models in fuzzy multipurpose decision making based on fuzzy preference relations. Fuzzy Sets Syst. 97(1), 33–48 (1998)

    Article  MathSciNet  Google Scholar 

  15. Chiclana, F., Herrera, F., Herrera-Viedma, E.: Integrating multiplicative preference relations in a multipurpose decision-making model based on fuzzy preference relations. Fuzzy Sets Syst. 122(2), 277–291 (2001)

    Article  MathSciNet  Google Scholar 

  16. Corcoran, A., Gillanders, R.: Foreign direct investment and the ease of doing business. Rev. world Econ. 151, 103–126 (2015)

    Article  Google Scholar 

  17. Crawford, G., Williams, C.: A note on the analysis of subjective judgment matrices. J. Math. Psychol. 29(4), 387–405 (1985)

    Article  Google Scholar 

  18. Dialga, I., Giang, T.H., L: Highlighting methodological limitations in the steps of composite indicators construction. Soc. Indic. Res. 131, 441–465 (2017)

    Article  Google Scholar 

  19. Djankov, S.: The doing business project: how it started: correspondence. J. Econ. Perspect. 30(1), 247–248 (2016)

    Article  Google Scholar 

  20. Ekel, P., Pedrycz, W., Pereira, J., Jr.: Multicriteria Decision-Making Under Conditions of Uncertainty: A Fuzzy set Perspective. John Wiley & Sons, Hoboken (2019)

    Book  Google Scholar 

  21. Ekel, P., Bernardes, P., Vale, G.M.V., Libório, M.P.: South American business environment cost index: Reforms for Brazil. Int. J. Bus. Environ. 13(2), 212–233 (2022a)

    Article  Google Scholar 

  22. Ekel, P., Bernardes, P., Laudares, S., Libório, M.P.: Evidence of the negative relationship between transaction costs and economic performance in G7 + BRICS countries. Technol. Audit Prod. Reserves. 5(67), 37–42 (2022b)

    Article  Google Scholar 

  23. El Gibari, S., Gómez, T., Ruiz, F.: Building composite indicators using multicriteria methods: a review. J. Bus. Econ. 89(1), 1–24 (2019)

    Google Scholar 

  24. Ergu, D., Kou, G., Peng, Y., Shi, Y.: A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP. Eur. J. Oper. Res. 213(1), 246–259 (2011)

    Article  MathSciNet  Google Scholar 

  25. Figueiredo, L.R., Frej, E.A., Soares, G.L., Ekel, P.Y.: Group decision-based construction of scenarios for multicriteria analysis in conditions of uncertainty on the basis of quantitative and qualitative information. Group Decis. Negot. 30(3), 665–696 (2021)

    Article  Google Scholar 

  26. Fusco, E.: Enhancing non-compensatory composite indicators: a directional proposal. Eur. J. Oper. Res. 242(2), 620–630 (2015a)

    Article  MathSciNet  Google Scholar 

  27. Fusco, E.: Potential improvements approach in composite indicators construction: the multi-directional benefit of the doubt model. Socio-Economic Plann. Sci. 85, 101447 (2023)

    Article  Google Scholar 

  28. Greco, S., Ishizaka, A., Tasiou, M., Torrisi, G.: On the methodological framework of composite indices: a review of the issues of weighting, aggregation, and robustness. Soc. Indic. Res. 141(1), 61–94 (2019)

    Article  Google Scholar 

  29. Herrera-Viedma, E., Herrera, F., Chiclana, F., Luque, M.: Some issues on consistency of fuzzy preference relations. Eur. J. Oper. Res. 154(1), 98–109 (2004)

    Article  MathSciNet  Google Scholar 

  30. Ishizaka, A., Lusti, M.: An expert module to improve the consistency of AHP matrices. Int. Trans. Oper. Res. 11(1), 97–105 (2004)

    Article  Google Scholar 

  31. Koczkodaj, W.W.: A new definition of consistency of pairwise comparisons. Math. Comput. Model. 18(7), 79–84 (1993)

    Article  Google Scholar 

  32. Kuc-Czarnecka, M., Piano, S.L., Saltelli, A.: Quantitative storytelling in the making of a composite indicator. Soc. Indic. Res. 149(3), 775–802 (2020)

    Article  Google Scholar 

  33. Lamata, M.T., Peláez, J.I.: A method for improving the consistency of judgements. Int. J. Uncertain. Fuzziness Knowledge-Based Syst. 10(06), 677–686 (2002)

    Article  MathSciNet  Google Scholar 

  34. Lee, S.K., Yu, J.H.: Composite indicator development using utility function and fuzzy theory. J. Oper. Res. Soc. 64(8), 1279–1290 (2013)

    Article  Google Scholar 

  35. Li, G., Kou, G., Peng, Y.: A group decision making model for integrating heterogeneous information. IEEE Trans. Syst. Man Cybern.: Syst. 48(6), 982–992 (2016)

    Article  Google Scholar 

  36. Libório, M.P., da Silva, L.M.L., Ekel, P.I., Figueiredo, L.R., Bernardes, P.: Consensus-based sub-indicator weighting approach: constructing composite indicators compatible with expert opinion. Soc. Indic. Res. 164, 1073–1099 (2022a)

    Article  Google Scholar 

  37. Libório, M.P., Ekel, P.Y., Martinuci, O.D.S., Figueiredo, L.R., Hadad, R.M., Lyrio, R.D.M., Bernardes, P.: Fuzzy set based intra-urban inequality indicator. Qual. Quant. 56(2), 667–687 (2022b)

    Article  Google Scholar 

  38. Libório, M.P., de Abreu, J.F., Ekel, P.I., Machado, A.M.C.: Effect of sub-indicator weighting schemes on the spatial dependence of multidimensional phenomena. J. Geogr. Syst. 25(2), 185–211 (2023)

    Article  Google Scholar 

  39. Munda, G., Nardo, M.: Constructing Consistent Composite Indicators: The Issue of Weights. Institute for the Protection and Security of the Citizen, Italy (2005)

    Google Scholar 

  40. Pearson, K.: On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin philosophical magazine J. Sci. 2(11), 559–572 (1901)

    Article  Google Scholar 

  41. Pedrycz, W., Ekel, P., Parreiras, R.: Fuzzy Multi-Criteria Decision-Making: Models, Methods and Applications. John Wiley & Sons, Hoboken (2011)

    Google Scholar 

  42. Pinheiro-Alves, R., Zambujal-Oliveira, J.: The ease of doing Business Index as a tool for investment location decisions. Econ. Lett. 117(1), 66–70 (2012)

    Article  Google Scholar 

  43. Proudlove, N.C., Goff, M., Walshe, K., Boaden, R.: The signal in the noise: robust detection of performance “outliers” in health services. J. Oper. Res. Soc. 70(7), 1102–1114 (2019)

    Article  Google Scholar 

  44. Ramalho, F.D., Ekel, P.Y., Pedrycz, W., Júnior, J.G.P., Soares, G.L.: Multicriteria decision making under conditions of uncertainty in application to multiobjective allocation of resources. Inform. Fusion. 49, 249–261 (2019)

    Article  Google Scholar 

  45. Ramalho, F.D., Silva, I.S., Ekel, P.Y., da Silva Martins, C.A.P., Bernardes, P., Libório, M.P.: Multimethod to prioritize projects evaluated in different formats. MethodsX. 8, 101371 (2021)

    Article  PubMed  PubMed Central  Google Scholar 

  46. Rogge, N.: Composite indicators as generalized benefit-of-the-doubt weighted averages. Eur. J. Oper. Res. 267(1), 381–392 (2018)

    Article  MathSciNet  Google Scholar 

  47. Saaty, T.L.: A scaling method for priorities in hierarchical structures. J. Math. Psychol. 15(3), 234–281 (1977)

    Article  MathSciNet  Google Scholar 

  48. Saaty, T.: Multicriteria Decision Making: The Analytic Hierarchy Process. McGraw-Hill, New York (1980)

    Google Scholar 

  49. Saisana, M., Tarantola, S.: State-of-the-art Report on Current Methodologies and Practices for Composite indicator Development. European Commission, Joint Research Centre, Institute for the Protection and the Security of the Citizen, Technological and Economic Risk Management Unit, Ispra, Italy (2002)

    Google Scholar 

  50. Salo, A.A., Hämäläinen, R.P.: On the measurement of preferences in the analytic hierarchy process. J. Multi-Criteria Decis. Anal. 6(6), 309–319 (1997)

    Article  Google Scholar 

  51. The World Bank Group:. DataBank: Doing Business 2020. Retrieved on May 5, 2022 (2020). From: https://databank.worldbank.org/source/doing-business

  52. World Bank Group, World Bank Group to Discontinue Doing Business Report:. Retrieved on April 25, 2023 (2021). From: https://www.worldbank.org/en/news/statement/2021/09/16/world-bank-group-to-discontinue-doing-business-report

  53. Wu, Z., Liao, H.: A consensus reaching process for large-scale group decision making with heterogeneous preference information. Int. J. Intell. Syst. 36(9), 4560–4591 (2021)

    Article  Google Scholar 

  54. Xia, M., Xu, Z., Chen, J.: Algorithms for improving consistency or consensus of reciprocal [0, 1]-valued preference relations. Fuzzy Sets Syst. 216, 108–133 (2013)

    Article  MathSciNet  Google Scholar 

  55. Zamani, K., Omrani, H.: A complete information PCA-imprecise DEA approach for constructing composite indicator with interval data: an application for finding development degree of cities. Int. J. Oper. Res. 44(4), 522–549 (2022)

    Article  Google Scholar 

  56. Zhang, Q., Chen, J.C., Chong, P.P.: Decision consolidation: criteria weight determination using multiple preference formats. Decis. Support Syst. 38(2), 247–258 (2004)

    Article  Google Scholar 

  57. Zhang, B., Dong, Y., Herrera-Viedma, E.: Group decision making with heterogeneous preference structures: an automatic mechanism to support consensus reaching. Group Decis. Negot. 28(3), 585–617 (2019)

    Article  Google Scholar 

  58. Zhou, M., Hu, M., Chen, Y.W., Cheng, B.Y., Wu, J., Herrera-Viedma, E.: Towards achieving consistent opinion fusion in group decision making with complete distributed preference relations. Knowl. Based Syst. 236, 107740 (2022)

    Article  Google Scholar 

Download references

Funding

This work was carried out with the support of the National Council for Scientific and Technological Development of Brazil (CNPq) - productivity grant 311922/2021-0 and Junior postdoctoral fellowship 151518/2022-0.

Author information

Authors and Affiliations

  1. Pontifical Catholic University of Minas Gerais, Belo Horizonte, Minas Gerais, 30535-012, Brazil

    Matheus Pereira Libório, Petr Iakovlevitch Ekel & Patrícia Bernardes

  2. Ibmec University Center, Rio de Janeiro, Rio de Janeiro, 20030-020, Brazil

    Luiz Flávio Autran Monteiro Gomes

  3. Enacom Handcrafted Technologies, Belo Horizonte, Minas Gerais, 31275-100, Brazil

    Matheus Pereira Libório & Douglas Alexandre Gomes Vieira

  4. Federal Center for Technological Education of Minas Gerais, Belo Horizonte, 30510-000, Brazil

    Douglas Alexandre Gomes Vieira

Authors
  1. Matheus Pereira Libório
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petr Iakovlevitch Ekel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrícia Bernardes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luiz Flávio Autran Monteiro Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Douglas Alexandre Gomes Vieira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matheus Pereira Libório.

Ethics declarations

Conflict of interest

The authors declare they have no financial interests.

Human and/or animals rights

No human participants and/or animals are involved in this research.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Libório, M.P., Ekel, P.I., Bernardes, P. et al. Specialists’ knowledge and cognitive stress in making pairwise comparisons. OPSEARCH 61, 51–70 (2024). https://doi.org/10.1007/s12597-023-00689-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12597-023-00689-2

Keywords

JEL Classification

Search

Navigation