Fuzzy Multi-criteria Decision Making Methods Applied to Usability Software Assessment: An Annotated Bibliography

  • Yamilis Fernández-Pérez
  • Ailyn Febles-Estrada
  • Carlos Cruz
  • José Luis Verdegay
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 125)


The software impacts every day in all aspects of our life. Therefore it is essential to assess the impact on the sustainability of any aspect of its application. A conscious and responsible use of resources is needed. One feature that is associated with the term sustainable software is the usability. It is very usual to define usability as a software ease of use, but this definition is ambiguous. For this reason, there are several definitions according to different approaches to measure it. In the last decades, a lot of papers have been published about software usability assessment models. These models have evolved from the use of algorithms, conventional statistics techniques, and Soft Computing methods. The goal of this work is to offer a collection of annotated bibliography about the characteristics, attributes and metrics used in usability assessment, focused in the Soft Computing techniques.



This research has been supported by projects TIN2014-55024-P from the Spanish Ministry of Economy and Competitiveness, and P11-TIC-8001 from the Andalusian Government (both including FEDER funds).


  1. 1.
    Ackerman, R., Parush, A., Nassar, F., Shtub, A.: Metacognition and system usability: Incorporating metacognitive research paradigm into usability testing. Comput. Hum. Behav. 54, 101–113 (2016)CrossRefGoogle Scholar
  2. 2.
    Al-Jamimi, H.A., Ahmed, M.: Prediction of software maintainability using fuzzy logic. In: IEEE 3rd International Conference Software Engineering and Service Science (ICSESS), pp. 702–705 (2012)Google Scholar
  3. 3.
    Basto Cordero, L.J., Ribeiro Parente Filho, L.F., Costa dos Santos, R., Gassenferth, W., Soares Machado, M.A.: Ipod system’s usability: an application of the fuzzy logic. Glob. J. Comput. Sci. Technol. 13(6) (2013)Google Scholar
  4. 4.
    Bertoa, M.F., Troya, J.M., Vallecillo, A.: Measuring the usability of software components. J. Syst. Softw. 79(3), 427–439 (2006)CrossRefGoogle Scholar
  5. 5.
    Bevan, N., Macleod, M.: Usability measurement in context. Behav. nf. Technol. 13(1–2), 132–145 (1994)CrossRefGoogle Scholar
  6. 6.
    Beyan, N.: The MUSIC methodology for usability measurement. Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems, pp. 123–124. ACM (1992)Google Scholar
  7. 7.
    Bhatnagar, S., Dubey, S.K., Rana, A.: Quantifying website usability using fuzzy approach. Int. J. Soft Comput. Eng. (IJSCE) 2(2), 424–428 (2012). ISSN: 2231-2307Google Scholar
  8. 8.
    Challa, J.S., Paul, A., Dada, Y., Nerella, V.: Integrated software quality evaluation: a fuzzy multi-criteria approach. J. Inf. Process. Syst. 7(3), 473–518 (2011)CrossRefGoogle Scholar
  9. 9.
    Challa, J.S., Paul, A., Dada, Y., Nerella, V., Srivastava, P.R.: Quantification of software quality parameters using fuzzy multi criteria approach. In: 2011 International Conference on Process Automation, Control and Computing, pp. 1–6, July 2011Google Scholar
  10. 10.
    Chang, E., Dillon, T.: A usability-evaluation metric based on a soft-computing approach. IEEE Trans. Syst. Man Cybern.— Part A: Syst. Hum. 36(2), 356–372 (2006)CrossRefGoogle Scholar
  11. 11.
    Chang, E.J., Dillon, T.S., Cook, D.: An intelligent system based usability evaluation metric. In: IASTED International Conference on Intelligent Information Systems, pp. 218–226. IEEE Computer Society (1997)Google Scholar
  12. 12.
    Costa, R., Augusta, M., Machado, S., Santos, D.J., Tavares, R., Campos, C.D.: Uma Aplicação da Matemática Nebulosa na Usabilidade de Pacotes Estatísticos. RESI Revista Eletrônica de Sistemas de Informação 10(1), 1–8 (2007)Google Scholar
  13. 13.
    Danfeng, Y., Fangchun, Y.: Fuzzy evaluation of sla-oriented QoSM (the quality of service management) in NGN. In: Proceedings of IC-BNMT2009 (2009)Google Scholar
  14. 14.
    Dasso, A., Funes, A.: Software quality metrics aggregation. In: 13th Argentine Symposium on Software Engineering, ASSE 2012, pp. 312–323 (2012). ISSN: 1850-2792Google Scholar
  15. 15.
    Dubey, S.K., Gulati, A., Rana, A.: Usability evaluation of software systems using fuzzy multi- criteria approach. IJCSI Int. J. Comput. Sci. Issues ISSN (Online) 1694–0814, 9(3), 404–409 (2012)Google Scholar
  16. 16.
    Dubey, S.K., Mittal, A., Rana, A.: Measurement of object oriented software usability using fuzzy AHP. Int. J. Comput. Sci. Telecommun. 3(5), 98–104 (2012)Google Scholar
  17. 17.
    Etaati, M.L., Sadi-Nezhad, S., Using, A.: Fuzzy analytical network process and ISO 9126 quality model in software selection: a case study in E-learnig systems. J. Appl. Sci. 11(1), 96–103 (2011)CrossRefGoogle Scholar
  18. 18.
    Folmer, E., Bosch, J.: Architecting for usability: a survey. J. Syst. Softw. 70(1–2), 61–78 (2004)CrossRefGoogle Scholar
  19. 19.
    Folmer, E., van Gurp, J., Bosch, J.: A framework for capturing the relationship between usability and software architecture. Softw. Process Improv. Pract. 8(2), 67–87 (2003)CrossRefGoogle Scholar
  20. 20.
    Furer, B., Ruggeri, F., Voci, S.M., Borges, C.A., Slater, B.: Avaliação da usabilidade de um sistema computadorizado de epidemiologia nutricional. Bras Epidemiol 16(4), 966–975 (2013)CrossRefGoogle Scholar
  21. 21.
    Gray, A.R., MacDonell, S.G.: Fuzzy logic for software metric models throughout the development life-cycle. Inf. Sci. Discuss. Pap. Ser. 99(20) (1999)Google Scholar
  22. 22.
    Harper, B.D., Norman, K.L.: Improving user satisfaction: the questionnaire for user interaction satisfaction version 5.5. In: Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, pp. 224–228 (1993)Google Scholar
  23. 23.
    Hedegaard, S., Simonsen, J.: Mining until it hurts: automatic extraction of usability issues from online reviews compared to traditional usability evaluation. In: Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational—NordiCHI ’14, pp. 157–166 (2014)Google Scholar
  24. 24.
    Hornbæk, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum.-Comput. Stud. 64(2), 79–102 (2006)CrossRefGoogle Scholar
  25. 25.
    Hub, M., Zatloukal, M.: Usability evaluation of selected web portals 2 case study usability evaluation of selected web portals. In: Proceedings of the 9th WSEAS International Conference on Applied Informatics and Communications (AIC ’09), pp. 259–264 (2009)Google Scholar
  26. 26.
    Hub, M., Zatloukal, M.: Model of usability evaluation of web portals based on the fuzzy logic. WSEAS Trans. Inf. Sci. Appl. 7(4), 522–531 (2010)Google Scholar
  27. 27.
    Jain, P., Dubey, S.K., Rana, A.: Software usability evaluation method. Int. J. Adv. Res. Comput. Eng. Technol. 1(2), 28–33 (2012)Google Scholar
  28. 28.
    Jin, B.S., Ji, Y.G.: Development of a usability evaluation framework with quality function deployment : from customer sensibility to product design. Hum. Factors Ergon. Manuf. 19(2), 177–194 (2009)CrossRefGoogle Scholar
  29. 29.
    Juristo, N., Moreno, A.M., Sanchez-Segura, M.-I.: Analysing the impact of usability on software design. J. Syst. Softw. 80(9), 1506–1516, Sept 2007Google Scholar
  30. 30.
    Komogortsev, O.V., Mueller, C.J., Tamir, D., Feldman, L.: An effort based model of software usability. In: Proceedings of the International Conference on Software Engineering Theory and Practice (SETP-08), vol. (July), pp. 1–9 (2009)Google Scholar
  31. 31.
    Kortum, P., Peres, S.C.: The relationship between system effectiveness and subjective usability scores using the system usability scale. Int. J. Hum.-Comput. Interact. 30(Nov 2015), 575–584 (2014)Google Scholar
  32. 32.
    Kumar, V., Sharma, A., Kumar, R., Grover, P.S.: Quality aspects for component-based systems: a metrics based approach. Softw. Pract. Exp. 42(January), 1531–1548 (2012)CrossRefGoogle Scholar
  33. 33.
    Kurosu, M.: Human-computer interaction users and contexts. In: proceedings of the 17th international conference, HCI international 2015 Los Angeles, CA, USA, August 2–7, 2015, Part III. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9171, pp. 35–42 (2015)Google Scholar
  34. 34.
    Lamichhane, R., Meesad, P.: A usability evaluation for government websites of nepal using fuzzy AHP. In: The 7th International Conference on Computing and Information Technology IC2IT2011, pp. 99–104 (2011)Google Scholar
  35. 35.
    Li, Y.-X., Man, Z.-X.: A fuzzy comprehensive quality evaluation for the digitizing software of ethnic antiquarian resources. In: 2008 International Conference on Computer Science and Software Engineering, pp. 1271–1274 (2008)Google Scholar
  36. 36.
    Liang, L., Deng, X., Wang, Y.: Usability measurement using a fuzzy simulation approach. In: 2009 International Conference on Computer Modeling and Simulation, pp. 88–92, Feb 2009Google Scholar
  37. 37.
    Madhavan, R., Alagarsamy, K.: Usability evaluation of object oriented applications using learning curves. Int. J. Eng. Tech. Res. (IJETR) 2(2 ISSN: 2321-0869), 35–39 (2014)Google Scholar
  38. 38.
    Mansor, Z., Kasirun, Z.M., Yahya, S., Arshad, N.H.: The evaluation of webcost using software usability measurement inventory ( SUMI ). Int. J. Digit. Inf. Wirel. Commun. (IJDIWC) 2(2), 197–201 (2012)Google Scholar
  39. 39.
    Mich, L., Franch, M., Gaio, L.: Evaluating and designing web site quality. Multimed. IEEE 10(1), 34–43 (2003)CrossRefGoogle Scholar
  40. 40.
    Mittal, H., Sharma, M., Mittal, J.: Analysis and modelling of websites quality using fuzzy technique. In: 2012 Second International Conference on Advanced Computing & Communication Technologies, pp. 10–15, Jan 2012Google Scholar
  41. 41.
    Montazer, G.A., Saremi, H.Q.: An application of Type-2 fuzzy notions in website structures selection: utilizing extended TOPSIS method. Wseas Trans. Comput. 7(1), 8–15 (2008)Google Scholar
  42. 42.
    Nagpal, R., Mehrotra, D., Sharma, A., Bhatia, P.: ANFIS method for usability assessment of website of an educational institute. World Appl. Sci. J. 23(11), 1489–1498 (2013)Google Scholar
  43. 43.
    Nielsen, J.: Usability Engineering. Academic Press, San Diego, CA (1993)zbMATHGoogle Scholar
  44. 44.
    Oztekin, A., Kong, Z.J., Uysal, O.: UseLearn: a novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. Int. J. Ind. Ergon. 40(4), 455–469 (2010)CrossRefGoogle Scholar
  45. 45.
    Parvinder, A., Sandhu, S., Priyanka Kakkar, S.S.: A survey on software reusability. In: 2010 International Conference on Mechanical and Electrical Technology (ICMET 2010), pp. 769–773 (2010)Google Scholar
  46. 46.
    Patalano, S., Lanzotti, A., Del Giudice, D.M., Vitolo, F., Gerbino, S.: On the usability assessment of the graphical user interface related to a digital pattern software tool. Int. J. Interact. Des. Manuf. (IJIDeM) (Dii) (2015)Google Scholar
  47. 47.
    Pizzi, N.J.: Software quality prediction using fuzzy integration: a case study. Soft Comput. 12(1), 67–76, June 2007Google Scholar
  48. 48.
    Preece, J.: Sociability and usability in online communities: determining and measuring success. Behav. Inf. Technol. 2001 20(5), 347–356 (2001)Google Scholar
  49. 49.
    Roy, S., Pattnaik, P.K., Mall, R.: A quantitative approach to evaluate usability of academic websites based on human perception. Egypt. Inf. J. 15(3), 159–167 (2014)CrossRefGoogle Scholar
  50. 50.
    Santos, D.J., Tavares Carneiro de Campos, R., Parente Filho, L.F., Costa dos Santos, R., Soares Machado, M.A.: Usabilidade de sistemas: uma aplicação da lógica fuzzy na avaliação do PHSTAT (2007)Google Scholar
  51. 51.
    Schenkman, B.N., Jönsson, F.U.: Aesthetics and preferences of web pages. Behav. Inf. Technol. 19(5), 367–377 (2000)CrossRefGoogle Scholar
  52. 52.
    Sharma, V., Sharma, A.: Software usability assessment models and metrics : a survey. Int. KIET J. Softw. Commun. Technol. (IKJSCT) 1(1), 7–15 (2013)Google Scholar
  53. 53.
    Sicilia, M.A., Barriocanal, E.G., Calvo, T.: An inquiry-based method for Choquet integral-based aggregation of interface usability parameters. Kybernetika 39(5), 601–614 (2003)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Sicilia, M.A., García, E.: Modelling Interacting Web Usability Criteria through Fuzzy Measures. Web Engineering. Springer, Berlin (2003)zbMATHGoogle Scholar
  55. 55.
    Singh, A., Dubey, S.K.: Evaluation of usability using soft computing technique. Int. J. Sci. Eng. Res. 4(12), 162–166 (2013)Google Scholar
  56. 56.
    Sitar-Taut, D.-A., Stanca, L.-M., Buchmann, R., Lacurezeanu, R.: A case study on usability metrics applied in romanian E-Commerce environment. Wseas Trans. Inf. Sci. Appl. 6(10), 1697–1706 (2009)Google Scholar
  57. 57.
    Siu Keung, W., Thi Nguyen, T., Chang, E., Jayaratna, N.: Usability metrics for E-learning. In: On The Move to Meaningful Internet Systems 2003: OTM 2003 Workshops, pp. 235–252. Springer, Berlin (2003)Google Scholar
  58. 58.
    Tamir, D., Komogortsev, O.V., Mueller, C.J.: An effort and time based measure of usability. In: Proceedings of the 6th International Workshop on Software Quality, pp. 47–52. ACM (2008)Google Scholar
  59. 59.
    Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: Usability Professional Association Conference, pp. 1–12 (2004)Google Scholar
  60. 60.
    Winter, S., Wagner, S., Deissenboeck, F.: A comprehensive model of usability. Engineering Interactive Systems, pp. 106–122. Springer, Berlin (2008)Google Scholar
  61. 61.
    Zenebe, A., Norcio, A.F.: Evaluation framework for fuzzy theoretic-based recommender system. In: 11th International Conference on Human-Computer Interaction (To Appear), Las Vegas, Nevada, USA (2005)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Yamilis Fernández-Pérez
    • 1
  • Ailyn Febles-Estrada
    • 1
  • Carlos Cruz
    • 2
  • José Luis Verdegay
    • 2
  1. 1.Universidad de las Ciencias InformáticasHavanaCuba
  2. 2.Universidad de GranadaGranadaSpain

Personalised recommendations