Skip to main content

A Comparison Between Performing a Heuristic Evaluation Based on a Formal Process Using a System and the Traditional Way: A Case Study

  • Conference paper
  • First Online:
Design, User Experience, and Usability (HCII 2023)

Abstract

The heuristic evaluation belongs to the usability inspection methods and is considered one of the most popular methods since it allows the discovery of over 75% of the total usability problems involving only 3 to 5 usability experts. However, certain problems and challenges have been identified during their execution. One of the identified problems is the lack of validation scenarios that demonstrate the contributions and benefits of incorporating systems that provide support for heuristic evaluation. Although there are case studies that could support the problems described above, the literature has shown that these are quite scarce and, for the most part, are software products that are used as support tools for certain elements to be evaluated, such as, for example, visual clarity of the system, page links validations and readability. For this reason, since there are no validation scenarios that demonstrate the contributions and benefits of the automation of heuristic evaluations through software products, the result is that there is a low level of confidence in usability evaluators to use a system that supports the heuristic evaluation process, for so the inspection continues to be carried out manually. As a part of previous research, a software product was implemented to support the five steps (planning, training, evaluation, discussion, and report) of a previously selected formal process to perform heuristic evaluations. To test the previously developed software product, a case study that demonstrates the contributions and benefits of incorporating the web application developed as a support tool for heuristic evaluation was proposed, and it has been of the utmost importance since it has allowed performing a comparative analysis between the evaluation when is performed by using the selected formal process through a template contained in an MS Excel format document, and by using the web application developed as a result of the aforementioned previous research. The results demonstrated that a higher average result was obtained, for the team that used the implemented system, in all the established criteria. The interpretation of the results can be summarized with the fact that the system obtained a better perception by the team who used it than the results obtained by the team that used the evaluation template.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Huang, Z.: Usability of tourism websites: a case study of heuristic evaluation. New Rev. Hypermedia Multimed. 26, 55–91 (2020). https://doi.org/10.1080/13614568.2020.1771436

    Article  Google Scholar 

  2. Holzinger, A.: Usability engineering methods for software developers. Commun. ACM. 48, 71–74 (2005). https://doi.org/10.1145/1039539.1039541

    Article  Google Scholar 

  3. Paz Espinoza, F.A.: Método para la evaluación de usabilidad de sitios web transaccionales basado en el proceso de inspección heurística. Pontif. Univ. Católica del Perú. 275 (2018). file:///C:/Users/UTM-BIBLIOTECA/Downloads/PAZ_FREDDY_USABILIDAD_SITIOS_WEB_%20INSPECCI%C3%93N_HEUR%C3%8DSTICA.pdf

    Google Scholar 

  4. Ghazali, M., Sivaji, A., Abdollah, N., Khean, C.N.: Visual Clarity Checker (VC2) to support heuristic evaluation: to what extent does VC2 help evaluators? Proc. - 2016 4th Int. Conf. User Sci. Eng. i-USEr 2016. 182–187 (2017). https://doi.org/10.1109/IUSER.2016.7857957

  5. Ahmad, W.F.W., Sarlan, A., Ezekiel, A., Juanis, L.: Automated Heuristic Evaluator. J. Informatics Math. Sci. 8, 301–306 (2016)

    Google Scholar 

  6. Lecaros, A., Moquillaza, A., Falconi, F., Aguirre, J., Tapia, A., Paz, F.: Selection and modeling of a formal heuristic evaluation process through comparative analysis. In: Soares, M.M., Rosenzweig, E., and Marcus, A. (eds.) Design, User Experience, and Usability: UX Research, Design, and Assessment, pp. 28–46. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05897-4_3

  7. International Organization for Standardization: ISO 9241–210–2019. (2019)

    Google Scholar 

  8. Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1993)

    Book  MATH  Google Scholar 

  9. Paz, F., Paz, F.A., Pow-Sang, J.A., Collazos, C.: A formal protocol to conduct usability heuristic evaluations in the context of the software development process. Int. J. Eng. Technol. 7, 10–19 (2018). https://doi.org/10.14419/ijet.v7i2.28.12874

  10. Edeki, C.: Agile Unified Process. Int. J. Comput. Sci. Mob. Appl. IJCSMA. 1, 13–17 (2013)

    Google Scholar 

  11. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. Manag. Inf. Syst. 13, 319–339 (1989). https://doi.org/10.2307/249008

    Article  Google Scholar 

  12. Paz, F.: Heurísticas de usabilidad para sitios web transaccionales. Pontif. Univ, Católica del Perú (2013)

    Google Scholar 

Download references

Acknowledgments

This work is part of the research project “Virtualización del proceso de evaluación de experiencia de usuario de productos de software para escenarios de no presencialidad” (virtualization of the user experience evaluation process of software products for non-presential scenarios), developed by HCI-DUXAIT research group. HCI-DUXAIT is a research group that belongs to the PUCP (Pontificia Universidad Católica del Perú).

This work was funded by the Dirección de Fomento a la Investigación at the PUCP through grant 2021-C-0023.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adrián Lecaros .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lecaros, A., Moquillaza, A., Falconi, F., Aguirre, J., Tapia, A., Paz, F. (2023). A Comparison Between Performing a Heuristic Evaluation Based on a Formal Process Using a System and the Traditional Way: A Case Study. In: Marcus, A., Rosenzweig, E., Soares, M.M. (eds) Design, User Experience, and Usability. HCII 2023. Lecture Notes in Computer Science, vol 14032. Springer, Cham. https://doi.org/10.1007/978-3-031-35702-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35702-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35701-5

  • Online ISBN: 978-3-031-35702-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics