Skip to main content
SpringerLink
Log in

Selection and Modeling of a Formal Heuristic Evaluation Process Through Comparative Analysis

  • Conference paper
  • First Online:
Design, User Experience, and Usability: UX Research, Design, and Assessment (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13321))

Included in the following conference series:

Abstract

Due to the importance of usability, multiple usability evaluation methods have been proposed that help Human-Computer Interaction (HCI) specialists determine whether the interfaces of a software product are usable, easy to use, understandable, and attractive. Among these methods, the heuristic evaluation proposed by Jakob Nielsen is the one that stands out. Although Nielsen offers general guidelines for the execution of heuristic evaluations, very few authors of different studies present a formal agreement or process on how the evaluations should be carried out, which leads us to the problem of the absence of a comparative analysis that allows determining the most appropriate formal evaluation process to carry out heuristic inspections. To complement, some proposals found in the literature compare the results of the execution of heuristic evaluations, the definition of new heuristics, qualification forms, and the formalization of the complete process in 5 phases. Although these proposals contribute to the formalization of the heuristic evaluation process, the literature review has not provided a comparative analysis that allows determining which is the most appropriate, which could cause usability evaluators to interpret and carry out their own procedure, which would lead to inaccuracies in the results and increase the probability of improperly executing the inspection. The purpose of this study was to elaborate a comparative table of the formal processes; this allowed the grouping of the various studies found in the literature, to select a process, and thus, and to model by a BPMN tool the whole process. Finally, this process modeled was validated by expert judgement of HCI specialists.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Li, Y., Liu, C.: User experience research and analysis based on usability testing methods. In: Zhao, P., Ye, Z., Xu, M., Yang, Li., Zhang, L., Zhu, R. (eds.) Advances in Graphic Communication, Printing and Packaging Technology and Materials. LNEE, vol. 754, pp. 263–267. Springer, Singapore (2021). https://doi.org/10.1007/978-981-16-0503-1_39

    Chapter  Google Scholar 

  2. Nielsen, J.: Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems. pp. 413–414 (1994). DOI: https://doi.org/10.1145/259963.260531

  3. Lecaros, A., Paz, F., Moquillaza, A.: Challenges and opportunities on the application of heuristic evaluations: a systematic literature review. In: Soares, M.M., Rosenzweig, E., Marcus, A. (eds.) HCII 2021. LNCS, vol. 12779, pp. 242–261. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78221-4_17

    Chapter  Google Scholar 

  4. Goetz, M.: Modeling Workflow Patterns through a Control-flow perspective using BPMN and the BPM Modeler BizAgi. Bpmn. (2009)

    Google Scholar 

  5. International Organization for Standardization: ISO 9241–210–2019: (2019)

    Google Scholar 

  6. Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1993)

    Book  Google Scholar 

  7. Holzinger, A.: Usability engineering methods for software developers. Commun. ACM. 48, 71–74 (2005). https://doi.org/10.1145/1039539.1039541

    Article  Google Scholar 

  8. Kitchenham, B., Charters, S.: Guidelines for performing Systematic Literature Reviews in Software Engineering (2007)

    Google Scholar 

  9. Paz Espinoza, F.A.: Método para la evaluación de usabilidad de sitios web transaccionales basado en el proceso de inspección heurística [Doctoral Thesis]. Pontif. Univ. Católica del Perú. 275 (2018). http://hdl.handle.net/20.500.12404/9903

  10. Hasan, L., Morris, A.: Usability problem areas on key international and key arab e-commerce websites. J. Internet Commer. 16, 80–103 (2017). https://doi.org/10.1080/15332861.2017.1281706

    Article  Google Scholar 

  11. Davis, D., Jiang, S.: Usability evaluation of web-based interfaces for Type2 Diabetes Mellitus. In: IEOM 2015 - 5th International Conference Industrial Engineering Operations Management Proceeding (2015). https://doi.org/10.1109/IEOM.2015.7093713

  12. Lam, D., Sajjanhar, A.: Heuristic evaluations of cultural heritage websites. In: 2018 International Conference Digital Image Computing Techniques Appllications DICTA 2018 (2019). https://doi.org/10.1109/DICTA.2018.8615847

  13. Jiménez, C., Cid, H.A., Figueroa, I.: PROMETHEUS: procedural methodology for developing heuristics of usability. IEEE Lat. Am. Trans. 15, 541–549 (2017). https://doi.org/10.1109/TLA.2017.7867606

    Article  Google Scholar 

  14. Paz, F., Paz, F.A., Pow-Sang, J.A., Collazos, C.: A formal protocol to conduct usability heuristic evaluations in the context of the software development process. Int. J. Eng. Technol. 7(2.28), 10–19 (2018). https://doi.org/10.14419/ijet.v7i2.28.12874

  15. Maghfiroh, L.R.: Observation and heuristics evaluation of student web-based application of SIPADU-STIS. J. Phys. Conf. Ser. 1511, 0–10 (2020) https://doi.org/10.1088/1742-6596/1511/1/012019

  16. Pincay-Ponce, J., Caicedo-Ávila, V., Herrera-Tapia, J., Delgado-Muentes, W., Delgado-Franco, P.: Usabilidad en sitios web oficiales de las universidades del ecuador. Rev. Ibérica Sist. e Tecnol. Informação. 106–120 (2020)

    Google Scholar 

  17. Walsh, L., Hemsley, B., Allan, M., Adams, N., Balandin, S., Georgiou, A., Higgins, I., McCarthy, S., Hill, S.: The e-health literacy demands of Australia’s my health record: a heuristic evaluation of usability. Perspect. Heal. Inf. Manag. 14 (2017)

    Google Scholar 

  18. Díaz, J., Rusu, C., Collazos, C.A.: Experimental validation of a set of cultural-oriented usability heuristics: e-Commerce websites evaluation. Comput. Stand. Interfaces. 50, 160–178 (2017). https://doi.org/10.1016/j.csi.2016.09.013

    Article  Google Scholar 

  19. Chen, Y.N., Hwang, S.L.: A heuristic evaluation on the usability of health information websites. In: Bridging Research and Good Practices towards Patient Welfare - Proceedings of the 4th International Conference on HealthCare Systems Ergonomics and Patient Safety, HEPS 2014, pp. 109–116 (2015)

    Google Scholar 

  20. Kiourexidou, M., Antonopoulos, N., Kiourexidou, E., Piagkou, M., Kotsakis, R., Natsis, K.: Websites with multimedia content: A heuristic evaluation of the medical/anatomical museums. Multimodal Technol. Interact. 3(2), 42 (2019). https://doi.org/10.3390/mti3020042

  21. Yeratziotis, A., Zaphiris, P.: A Heuristic Evaluation for Deaf Web User Experience (HE4DWUX). Int. J. Hum. Comput. Interact. 34, 195–217 (2018). https://doi.org/10.1080/10447318.2017.1339940

    Article  Google Scholar 

  22. Eliseo, M.A., Casac, B.S., Gentil, G.R.: A comparative study of video content user interfaces based on heuristic evaluation. In: Iberian Conference Information Systems Technologies CISTI, pp. 1-6 (2017). https://doi.org/10.23919/CISTI.2017.7975820

  23. Motlagh Tehrani, S.E., Zainuddin, N.M.M., Takavar, T.: Heuristic evaluation for Virtual Museum on smartphone. In: Proceeding - 2014 3rd International Conference User Science Engineering Experimental Engineering Engag. i-USEr 2014, pp. 227–231 (2015) https://doi.org/10.1109/IUSER.2014.7002707

  24. Puspitasari, I., Indah Cahyani, D.: Taufik: a user-centered design for redesigning e-government website in public health sector. In: Proceeding - 2018 International Seminar Application Technology Information Communication Creat. Technology Human Life, iSemantic 2018, pp. 219–224 (2018) https://doi.org/10.1109/ISEMANTIC.2018.8549726

  25. Quinones, D., Rusu, C., Roncagliolo, S., Rusu, V., Collazos, C.A.: Developing usability heuristics: a formal or informal process? IEEE Lat. Am. Trans. 14, 3400–3409 (2016). https://doi.org/10.1109/TLA.2016.7587648

    Article  Google Scholar 

  26. Halaweh, M.: Usability testing of conferences websites: a case study of practical teaching. In: Uden, L., Hadzima, B., Ting, I.-H. (eds.) KMO 2018. CCIS, vol. 877, pp. 380–389. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95204-8_32

    Chapter  Google Scholar 

  27. Silvis, I.M., Bothma, T.J.D., de Beer, K.J.W.: Evaluating the usability of the information architecture of academic library websites. Libr. Hi Tech. 37, 566–590 (2019). https://doi.org/10.1108/LHT-07-2017-0151

    Article  Google Scholar 

  28. Paz, F., Paz, F.A., Arenas, J.J., Rosas, C.: A Perception study of a new set of usability heuristics for transactional web sites. In: Karwowski, W., Ahram, T. (eds.) IHSI 2018. AISC, vol. 722, pp. 620–625. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73888-8_96

    Chapter  Google Scholar 

  29. Paz, F., Paz, F.A., Pow-Sang, J.A.: Comparing the effectiveness and accuracy of new usability heuristics. Adv. Intell. Syst. Comput. 497, 163–175 (2017). https://doi.org/10.1007/978-3-319-41956-5_16

    Article  Google Scholar 

  30. Paz, F., Paz, F.A., Sánchez, M., Moquillaza, A., Collantes, L.: Quantifying the usability through a variant of the traditional heuristic evaluation process. In: Marcus, A., Wang, W. (eds.) DUXU 2018. LNCS, vol. 10918, pp. 496–508. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91797-9_36

    Chapter  Google Scholar 

  31. Inal, Y.: University students’ heuristic usability ınspection of the national library of Turkey website. Aslib J. Inf. Manag. 70, 66–77 (2018). https://doi.org/10.1108/AJIM-09-2017-0216

    Article  Google Scholar 

  32. dos Santos Pergentino, A.C., Canedo, E.D., Lima, F., de Mendonça, F.L.L.: Usability heuristics evaluation in search engine. In: Marcus, A., Rosenzweig, E. (eds.) HCII 2020. LNCS, vol. 12200, pp. 351–369. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49713-2_25

    Chapter  Google Scholar 

  33. Iadanza, E., Fabbri, R., Luschi, A., Melillo, P., Simonelli, F.: A collaborative RESTful cloud-based tool for management of chromatic pupillometry in a clinical trial. Heal. Technol. 10(1), 25–38 (2019). https://doi.org/10.1007/s12553-019-00362-z

    Article  Google Scholar 

  34. Federici, S., et al.: Heuristic evaluation of eglu-box: a semi-automatic usability evaluation tool for public administrations. In: Kurosu, M. (ed.) HCII 2019. LNCS, vol. 11566, pp. 75–86. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22646-6_6

    Chapter  Google Scholar 

  35. de Menezes, M., Falco, M.: The relationship of the studies of ergonomic and human computer interfaces – a case study of graphical interfaces in e-commerce websites. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11586, pp. 474–484. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23535-2_35

    Chapter  Google Scholar 

  36. Paz, F., Paz, F.A., Pow-Sang, J.A.: Evaluation of usability heuristics for transactionalweb sites: a comparative study. Adv. Intell. Syst. Comput. 448, 1063–1073 (2016) https://doi.org/10.1007/978-3-319-32467-8_92

  37. Retore, A.P., Guimarães, C., Leite, M.K.: Heuristics for grid and typography evaluation of art magazines websites. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9731, pp. 408–416. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39510-4_38

    Chapter  Google Scholar 

  38. Huang, Z.: Usability of tourism websites: a case study of heuristic evaluation. New Rev. Hypermedia Multimed. 26, 55–91 (2020). https://doi.org/10.1080/13614568.2020.1771436

    Article  Google Scholar 

  39. Morales, J., Rusu, C., Botella, F., Quiñones, D.: Programmer experience: a set of heuristics for programming environments. In: Meiselwitz, G. (ed.) HCII 2020. LNCS, vol. 12195, pp. 205–216. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49576-3_15

    Chapter  Google Scholar 

  40. Okhovati, M., Karami, F., Khajouei, R.: Exploring the usability of the central library websites of medical sciences universities. J. Librariansh. Inf. Sci. 49, 246–255 (2017). https://doi.org/10.1177/0961000616650932

    Article  Google Scholar 

  41. Fung, R.H.Y., Chiu, D.K.W., Ko, E.H.T., Ho, K.K.W., Lo, P.: Heuristic usability evaluation of university of hong kong libraries’ mobile website. J. Acad. Librariansh. 42, 581–594 (2016). https://doi.org/10.1016/j.acalib.2016.06.004

    Article  Google Scholar 

Download references

Acknowledgments

This work is part of the research project “Virtualización del proceso de evaluación de experiencia de usuario de productos de software para escenarios de no presencialidad” (virtualization of the user experience evaluation process of software products for non-presential scenarios), developed by HCI-DUXAIT research group. HCI-DUXAIT is a research group that belongs to the PUCP (Pontificia Universidad Católica del Perú).

This work was funded by the Dirección de Fomento a la Investigación at the PUCP through grant 2021-C-0023.

Author information

Authors and Affiliations

  1. Pontificia Universidad Católica del Perú, Av. Universitaria 1801, San Miguel, Lima 32, Lima, Perú

    Adrian Lecaros, Arturo Moquillaza, Fiorella Falconi, Joel Aguirre, Alejandro Tapia & Freddy Paz

Authors
  1. Adrian Lecaros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arturo Moquillaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fiorella Falconi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joel Aguirre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alejandro Tapia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Freddy Paz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian Lecaros .

Editor information

Editors and Affiliations

  1. Southern University of Science and Technology – SUSTech, Shenzhen, China

    Marcelo M. Soares

  2. World Usability Day and Bubble Mountain Consulting, Newton Center, MA, USA

    Elizabeth Rosenzweig

  3. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lecaros, A., Moquillaza, A., Falconi, F., Aguirre, J., Tapia, A., Paz, F. (2022). Selection and Modeling of a Formal Heuristic Evaluation Process Through Comparative Analysis. In: Soares, M.M., Rosenzweig, E., Marcus, A. (eds) Design, User Experience, and Usability: UX Research, Design, and Assessment. HCII 2022. Lecture Notes in Computer Science, vol 13321. Springer, Cham. https://doi.org/10.1007/978-3-031-05897-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05897-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05896-7

  • Online ISBN: 978-3-031-05897-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation