Skip to main content
SpringerLink
Log in

Analytic hierarchy process-based group assessment of quality-in-use model characteristics

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

While the characteristics related to the qualities of software in use have been standardized, people managing software development processes still have to combine and prioritize attainment levels for such characteristics. Ranking them can therefore be considered a decision problem that should be solved not only in accordance with the preferences of the stakeholders involved in the decision-making process, but also by following multi-part standards, e.g., that the relative importance of quality characteristics should depend on the high-level goals and objectives for the project. This paper presents an example guide for creating such a ranking for a group of experts coming from different domains using a decision-support inspired approach. Five important quality-in-use characteristics are evaluated by sixteen experts with the analytic hierarchy process. Obtained individual preferences were aggregated by two procedures, and achieved group results were analyzed, of which one analysis included testing of their conformity to individual results. For this case study, the group opinions indicated the top-valued quality in use was effectiveness, followed by satisfaction, freedom from risk, efficiency and context coverage in this ranked order (freedom from risk and efficiency were of nearly equal importance). Implications for the future work of applying different decision-making models such as social choice theory for studying quality attributes are also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Aczel, J., Saaty, T.L.: Procedures for synthesizing ratio judgments. Math. Psychol. 27, 93–102 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  2. Asahi, T., Turo, D., Shneiderman, B.: Using treemaps to visualize the analytic hierarchy process. Inform. Syst. Res. 6(4), 357–375 (1995)

    Article  Google Scholar 

  3. Barzilai, J.: Deriving weights from pair wise comparison matrices. J. Oper. Res. Soc. 48, 1226–1232 (1997)

    Article  MATH  Google Scholar 

  4. Chandran, B., Golden, B., Wasil, E.: Linear programming models for estimating weights in the analytic hierarchy process. Comput. Oper. Res. 32, 2235–2254 (2005)

    Article  MATH  Google Scholar 

  5. Crawford, G., Williams, C.: A note on the analysis of subjective judgment matrices. J. Math. Psychol. 29, 387–405 (1985)

    Article  MATH  Google Scholar 

  6. Dong, Y.C., Xu, Y.F., Yu, S.: Computing the numerical scale of the linguistic term set for the 2-tuple fuzzy linguistic representation model. IEEE Trans. Fuzzy Syst. 17, 1366–1378 (2009)

    Article  Google Scholar 

  7. Finnie, G.R., Wittig, G.E., Petkov, D.I.: Prioritizing software development productivity factors using the analytic hierarchy process. Syst. Softw. 22(2), 129–139 (1993)

    Article  Google Scholar 

  8. Forman, E., Peniwati, K.: Aggregating individual judgments and priorities with the analytic hierarchy process. Eur. J. Oper. Res. 108, 165–169 (1998)

    Article  MATH  Google Scholar 

  9. Gulfem, I., Gulcin, B.: Using a multi-criteria decision making approach to evaluate mobile phone alternatives. Comput. Stand. Interf. 29(2), 265–274 (2007)

    Article  Google Scholar 

  10. Harker, P., Vargas, L.: The theory of ratio scale estimation: Saaty’s analytic hierarchy process. Manage. Sci. 33, 1383–1403 (1987)

    Article  MathSciNet  Google Scholar 

  11. Hartmann, S., Martini, C., Sprenger, J.: Consensual decision-making among epistemic peers. Episteme 6, 110–129 (2009)

    Article  Google Scholar 

  12. Hornbæk, K.: Dogmas in the assessment of usability evaluation methods. Behav. Inform. Technol. 29(1), 97–111 (2010)

    Article  Google Scholar 

  13. Jung, H.W.: Validating the external quality sub characteristics of software products according to ISO/IEC 9126. Comput. Stand. Interf. 29(6), 653–661 (2007)

    Article  Google Scholar 

  14. Karlsson, J., Ryan, K.: A cost-value approach for prioritizing requirements. IEEE Softw. 14, 67–74 (1997)

    Article  Google Scholar 

  15. Kim, K., Proctor, R.W., Salvendy, G.: The relation between usability and product success in cell phones. Behav. Inform. Technol. 31(10), 969–982 (2012)

    Article  Google Scholar 

  16. Kumar, V.N., Ganesh, L.: A simulation-based evaluation of the approximate and the exact eigenvector methods employed in AHP. Eur. J. Oper. Res. 95, 656–662 (1996)

    Article  MATH  Google Scholar 

  17. Lee, K., Lee, S.J.: A quantitative evaluation model using the ISO/IEC 9126 quality model in the component based development process. Computational Science and Its Applications, pp. 917–926. Springer, Berlin (2006)

    Google Scholar 

  18. Lehrer, K., Wagner, C.: Rational Consensus in Science and Society. Reidel, Dordrecht (1981)

    Book  MATH  Google Scholar 

  19. Lin, H.F.: Determining the relative importance of mobile banking quality factors. Comput. Stand. Interf. 35(2), 195–204 (2013)

    Article  Google Scholar 

  20. Mikhailov, L.: A fuzzy programming method for deriving priorities in the analytic hierarchy process. J. Oper. Res. Soc. 51, 341–349 (2000)

    Article  MATH  Google Scholar 

  21. Regan, H.M., Colyvan, M., Markovchick-Nicholls, L.: A formal model for consensus and negotiation in environmental management. J. Environ. Manage. 80(2), 167–176 (2006)

    Article  Google Scholar 

  22. Saaty, T.L.: The analytic hierarchy process. McGraw-Hill Inc, New York (1980)

    MATH  Google Scholar 

  23. Salmeron, J.L., Herrero, I.: An AHP-based methodology to rank critical success factors of executive information systems. Comput. Stand. Interf. 28(1), 1–12 (2005)

    Article  Google Scholar 

  24. Srdjevic, B.: Combining different prioritization methods in analytic hierarchy process synthesis. Comput. Oper. Res. 32, 1897–1919 (2005)

    Article  MATH  Google Scholar 

  25. Srdjevic, B., Srdjevic, Z.: Bi-criteria evolution strategy in estimating weights from the AHP ratio-scale matrices. Appl. Math. Comput. 218, 1254–1266 (2011)

    MathSciNet  MATH  Google Scholar 

  26. Srdjevic, B., Srdjevic, Z.: Synthesis of individual best local priority vectors in AHP-group decision making. Appl. Soft Comput. 13, 2045–2056 (2013)

    Article  MATH  Google Scholar 

  27. Srdjevic, B., Pipan, M., Srdjevic, Z., Arh, T.: AHP supported evaluation of LMS quality. In Proceedings of the International Workshop on the Interplay Between User Experience Evaluation and system Development (I-UxSED 2012), NORDCHI. Copenhagen, Denmark, pp. 52–57, (2012)

  28. Takeda, E., Cogger, K., Yu, P.L.: Estimating criterion weights using eigenvectors: a comparative study. Eur. J. Oper. Res. 29, 360–369 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  29. Tossell, C.C., Kortum, P., Shepard, C., Rahmati, A., Zhong, L.: An empirical analysis of smartphone personalisation: measurement and user variability. Behav. Inform. Technol. 31(10), 995–1010 (2012)

    Article  Google Scholar 

  30. Triantaphyllou, E., Mann, S.H.: Using the analytic hierarchy process for decision making in engineering applications: some challenges. Ind. Eng. Appl. Pract. 2(1), 35–44 (1995)

    Google Scholar 

  31. United Nation Industrial Development Organization (UNIDO) (2006) Role of standards: a Guide for Small and Medium-sized Enterprises. Accessible at: http://www.unido.org/index.php?id=o71886

  32. Wang, C.H., Hsueh, O.Z.: A novel approach to incorporate customer preference and perception into product configuration: a case study on smart pads. Comput. Stand. Interf. 35(5), 549–556 (2013)

    Article  Google Scholar 

  33. Winckler, M., Bach, C., Bernhaupt, R.: Identifying User eXperiencing factors along the development process: A case study. In Proceedings of the International Workshop on the Interplay Between User Experience Evaluation and System Development (I-UxSED 2012), NORDCHI, Copenhagen, Denmark, pp 37–42, (2012)

  34. Zahedi, F.: A simulation study of estimation methods in the analytic hierarchy process. Socio Econ. Plann. Sci. 6, 347–354 (1986)

    Article  Google Scholar 

  35. Zhang, T., Rau, P.L.P., Salvendy, G.: Exploring critical usability factors for handsets. Behav. Inform. Technol. 29(1), 45–55 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

This work is a part of common research within the framework of the COST Action IC0904 Towards the Integration of Transectorial IT Design and Evaluation (TwinTide). It was also supported in part by the Serbian Ministry of Education, Science and Technological Development and Secretariat for Science and Technological Development of Vojvodina Province.

Author information

Authors and Affiliations

  1. Faculty of Agriculture, University of Novi Sad, Novi Sad, Serbia

    Bojan Srđević

  2. Lab for Open Systems and Networks, Jožef Stefan Institute, Ljubljana, Slovenia

    Matija Pipan

  3. Faculty of Economics, University of Coimbra, Coimbra, Portugal

    Paulo Melo

  4. Department of Computer Science, University of Leicester, Leicester, UK

    Effie Lai-Chong Law

Authors
  1. Bojan Srđević
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matija Pipan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paulo Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Effie Lai-Chong Law
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bojan Srđević.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srđević, B., Pipan, M., Melo, P. et al. Analytic hierarchy process-based group assessment of quality-in-use model characteristics. Univ Access Inf Soc 15, 473–483 (2016). https://doi.org/10.1007/s10209-015-0421-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-015-0421-4

Keywords

Search

Navigation