Consistency-Driven Pairwise Comparisons Approach to Software Product Management and Quality Measurement

  • Waldemar W. Koczkodaj
  • Paweł Dymora
  • Mirosław Mazurek
  • Dominik StrzałkaEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 761)


In this study, the software product quality measurement, based on the consistency-drive pairwise comparisons (PC) is proposed as a new way of approaching this complicated problem. The assessment of software quality (SQ) is a complex process. It is usually done by experts who use their knowledge and experience. Their subjective assessments certainly involve inaccuracy (which is difficult to control) and consistency of assessments (which can be measured and may influence accuracy). The inconsistency analysis, which is proposed in this approach, is used to improve assessments. Weights, reflecting the relative importance of the attributes, are computed as opposed to the commonly practiced arbitrary assignment. The PC method allows to define a consistency measure and use it as a validation technique. A consistency-driven knowledge acquisition, supported by a properly designed software, contributes to the improvement of quality of knowledge-based systems.


Software quality Pairwise comparison Complex system Concluder 


  1. 1.
    Chemuturi, M.: Mastering Software Quality Assurance: Best Practices. Tools and Technique for Software Developers. J. Ross Publishing, Fort Lauderdale (2010)Google Scholar
  2. 2.
    Pressman, S.: Software Engineering: A Practitioner’s Approach, 6th edn. McGraw-Hill Education Pressman, Boston (2005)zbMATHGoogle Scholar
  3. 3.
    Zhang, X.M., Teng, X.L., Pham, H.: Considering fault removal efficiency in software reliability assessment. IEEE Trans. Syst. Man Cybern. Syst. Hum. 33(1), 114–119 (2003)CrossRefGoogle Scholar
  4. 4.
    Khoshgoftaar, T.M., Bhattacharyya, B.B., Richardson, G.D.: Predicting software errors, during development, using nonlinear regression models: a comparative study. IEEE Trans. Reliab. 41(3), 390–395 (1992)CrossRefGoogle Scholar
  5. 5.
    Kenny, G.Q.: Estimating defects in commercial software during operational use. IEEE Trans. Reliab. 42(1), 107–115 (1993)CrossRefGoogle Scholar
  6. 6.
    Koczkodaj, W.W.: Testing the accuracy enhancement of pairwise comparisons by a Monte Carlo experiment. J. Stat. Plan. Infer. 69(1), 21–31 (1998)CrossRefGoogle Scholar
  7. 7.
    Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. 140, 1–55 (1932)Google Scholar
  8. 8.
    Faliszewski, P., Hemaspaandra, E., Hemaspaandra, L.A., Rothe, J.: Llull and Copeland voting computationally resist bribery and constructive control. J. Artif. Intell. Res. 35, 275–341 (2009). Conference: 2nd International Workshop on Computational Social Choice Location: Liverpool, EnglandMathSciNetzbMATHGoogle Scholar
  9. 9.
    Thurstone, L.L.: A law of comparative assessments. Psychol. Rev. 34, 273–286 (1927)CrossRefGoogle Scholar
  10. 10.
    Colonius, H.: Representation and uniqueness of the Bradley-Terry-Luce model for pair comparisons. Br. J. Math. Stat. Psychol. 33, 99–103 (1980)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Saaty, T.L.: A scaling methods for priorities in hierarchical structure. J. Math. Psychol. 15, 234–281 (1977)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Koczkodaj, W.W., Mikhailov, L., Redlarski, G., Soltys, M., Szybowski, J., Tamazian, G., Wajch, E., Yuen, K.K.F.: Important facts and observations about pairwise comparisons. Fundamenta Informaticae 144, 1–17 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Koczkodaj, W.W.: Pairwise comparisons rating scale paradox. In: Transactions on Computational Collective Intelligence XXII. Lecture Notes in Computer Science, vol. 9655, pp. 1–9 (2016)Google Scholar
  14. 14.
    Williams, C., Crawford, G.: Analysis of subjective judgement matrices. The Rand Corporation Report R-2572-AF, pp. 1–59 (1980)Google Scholar
  15. 15.
    Koczkodaj, W.W.: Statistically accurate evidence of improved error rate by pairwise comparisons. Percept. Mot. Skills 82(1), 43–48 (1996)CrossRefGoogle Scholar
  16. 16.
    Koczkodaj, W.W., Szwarc, R.: Pairwise comparisons simplified. Appl. Math. Comput. 253, 387–394 (2015)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Jensen, R.: An alternative scaling method for priorities in hierarchical structures. J. Math. Psychol. 28, 317–332 (1984)CrossRefGoogle Scholar
  18. 18.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38(3), 367–426 (1996)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Bozoki, S., Fulop, J., Koczkodaj, W.W.: An LP-based inconsistency monitoring of pairwise comparison matrices. Math. Compute Model. 54(1–2), 789–793 (2011)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Khoshgoftaar, T.M., Allen, E.B., Bullard, L.A., Halstead, R., Trio, G.P.: A tree-based classification model for analysis of a military software system. In: Proceedings of IEEE High-Assurance Systems Engineering Workshop, pp. 244–251 (1997)Google Scholar
  21. 21.
    IEEE Standard 1061-1992: Standard for a Software Quality Metrics Methodology. Institute of Electrical and Electronics Engineers, New York (1992)Google Scholar
  22. 22.
    McCall, J., Richards, P., Walters, G.: Factors in Software Quality, NTIS (1977)Google Scholar
  23. 23.
    Boehm, B.W., Brown, J.R., Lipow, M.: Quantitative evaluation of software quality. In: Proceedings of the 2nd International Conference on Software Engineering. IEEE Computer Society Press (1976)Google Scholar
  24. 24.
    Leffingwell, D., Widrig, D.: Managing Software Requirements: A Use Case Approach, 2nd edn. Addison Wesley, Boston (2003)Google Scholar
  25. 25.
    The Rome Air Development Center is now the Rome Laboratory, as of 1991Google Scholar
  26. 26.
    Wiegers, K.: Software Requirements, 2nd edn. Microsoft Press, Redmond (2003)Google Scholar
  27. 27.
    Wohlin, C., Lundberg, L., Mattsson, M.: Special issue: trade-off analysis of software quality attributes. Softw. Qual. J. 13, 327–328 (2005)CrossRefGoogle Scholar
  28. 28.
    Sommerville, I.: Software Engineering, 9th edn. Addison-Wesley Longman Publishing Co., Inc., Boston (2006)zbMATHGoogle Scholar
  29. 29.
    Boehm, B.W., Brown, J.R., Kaspar, H., Lipow, M., Macleod, G., Merrit, M.: Characteristics of Software Quality. North-Holland, Amsterdam (1978)zbMATHGoogle Scholar
  30. 30.
    Pfleeger, S.L., Atlee, J.M.: Software Engineering: Theory and Practice. Prentice Hall PTR, Upper Saddle River (2009)Google Scholar
  31. 31.
  32. 32.
    Heer, J., Card, S.K., Landay, J.A.: Prefuse: a toolkit for interactive information visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Portland, Oregon, USA, pp. 421–430. ACM (2005)Google Scholar
  33. 33.
    The Agile Movement., see also W. W. Royce, Managing the development of large software systems: concepts and techniques, Proceeding ICSE 1987, pp. 328–338 (1987)
  34. 34.
    Duncan, G.O.: Private communication. Accessed 02 Mar 2017Google Scholar
  35. 35.
    Lessmann, S., Baesens, B., Mues, C., Pietsch, S.: Benchmarking classification models for software defect prediction: a proposed framework and novel findings. IEEE Trans. Softw. Eng. 34(4), 485–496 (2008)CrossRefGoogle Scholar
  36. 36.
  37. 37.
    ISO/IEC 25010:2011, Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models (2011)Google Scholar
  38. 38.
    Feigenbaum, A.V.: Total Quality Control. McGraw-Hill, New York (1983)Google Scholar
  39. 39.
    Kitchenham, B., Pfleeger, S.L.: Software quality: the elusive target. IEEE Softw. 13(1), 12–21 (1996)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Waldemar W. Koczkodaj
    • 1
  • Paweł Dymora
    • 2
  • Mirosław Mazurek
    • 2
  • Dominik Strzałka
    • 2
    Email author
  1. 1.Computer ScienceLaurentian UniversitySudburyCanada
  2. 2.Faculty of Electrical and Computer EngineeringRzeszów University of TechnologyRzeszówPoland

Personalised recommendations