Skip to main content
SpringerLink
Log in

Heuristic usability evaluation on games: a modular approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Heuristic evaluation is the preferred method to assess usability in games when experts conduct this evaluation. Many heuristics guidelines have been proposed attending to specificities of games but they only focus on specific subsets of games or platforms. In fact, to date the most used guideline to evaluate games usability is still Nielsen’s proposal, which is focused on generic software. As a result, most evaluations do not cover important aspects in games such as mobility, multiplayer interactions, enjoyability and playability, etc. To promote the usage of new heuristics adapted to different game and platform aspects we propose a modular approach based on the classification of existing game heuristics using metadata and a tool, MUSE (Meta-heUristics uSability Evaluation tool) for games, which allows a rebuild of heuristic guidelines based on metadata selection in order to obtain a customized list for every real evaluation case. The usage of these new rebuilt heuristic guidelines allows an explicit attendance to a wide range of usability aspects in games and a better detection of usability issues. We preliminarily evaluate MUSE with an analysis of two different games, using both the Nielsen’s heuristics and the customized heuristic lists generated by our tool.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Apperley TH (2006) Genre and game studies: Toward a critical approach to video game genres. Simul Gaming 37(1):6–23

    Article  Google Scholar 

  2. Barendregt W, Bekker MM, Bouwhuis DG, Baauw E (2006) Identifying usability and fun problems in a computer game during first use and after some practice. Int J Human-Comput Stud 64(9):830–846

    Article  Google Scholar 

  3. Brooke J (1996) SUS-A quick and dirty usability scale. Usab Eval Indust 189(194):4–7

    Google Scholar 

  4. Brown E, Cairns P (2004) A grounded investigation of game immersion. In CHI'04 extended abstracts on Hum Factors in Comput Syst (pp. 1297-1300). ACM

  5. Caillois R (1961) Man, play, and games. University of Illinois Press

  6. Cheng K, Cairns PA (2005) Behaviour, realism and immersion in games. In CHI'05 extended abstracts on Hum Factors in Comput Syst (pp. 1272-1275). ACM

  7. Clanton, C. (1998). An interpreted demonstration of computer game design. In CHI 98 conference summary on Hum Factors in Comput Syst (pp. 1-2). ACM

  8. Cowley B, Charles D, Black M, Hickey R (2008) Toward an understanding of flow in video games. Comput Entertain (CIE) 6(2):20

    Google Scholar 

  9. Csikszentmihalyi M (1975) Play and intrinsic rewards. Journal of humanistic psychology

  10. Dempsey JV, Lucassen BA, Haynes LL, Casey MS (1997) An exploratory study of forty computer games. University of South Alabama, Mobile, AL

    Google Scholar 

  11. Desurvire H, Wiberg C (2009) Game usability heuristics (PLAY) for evaluating and designing better games: The next iteration. In International Conference on Online Communities and Social Computing (pp. 557-566). Springer, Berlin, Heidelberg

  12. Desurvire H, Caplan M, Toth JA (2004). Using heuristics to evaluate the playability of games In CHI'04 extended abstracts on Hum Factors in Comput Syst (pp. 1509-1512). ACM

  13. FDIs I. S. O (2009) 9241-210: 2009. Ergonomics of human system interaction-Part 210: Human-centered design for interactive systems (formerly known as 13407). International Organization for Standardization (ISO). Switzerland

  14. Federoff MA (2002). Heuristics and usability guidelines for the creation and evaluation of fun in video games (Doctoral dissertation, Indiana University)

  15. Frøkjær E, Hertzum M, Hornbæk K (2000) Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 345-352). ACM

  16. Garzotto F (2007) Investigating the educational effectiveness of multiplayer online games for children. In Proceedings of the 6th international conference on Interaction design and children (pp. 29-36). ACM

  17. Grim Fandango Remastered (2015) http://www.grimremastered.com/ Lucasfilm Ltd. Retrieved in November March 2017

  18. Hassenzahl M (2008) User experience (UX): towards an experiential perspective on product quality. In Proceedings of the 20th Conference on l'Interaction Homme-Machine (pp. 11-15). ACM

  19. Hazlett RL (2006) Measuring emotional valence during interactive experiences: boys at video game play. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 1023-1026). ACM

  20. Hertzum M, Jacobsen NE (2001) The evaluator effect: A chilling fact about usability evaluation methods. Int J Human-Comput Interact 13(4):421–443

    Article  Google Scholar 

  21. Hwang W, Salvendy G (2010) Number of people required for usability evaluation: the 10 ± 2 rule. Commun ACM 53(5):130–133

    Article  Google Scholar 

  22. IJsselsteijn W, De Kort Y, Poels K, Jurgelionis A, Bellotti F (2007) Characterising and measuring user experiences in digital games. Int Conf Adv Comput Entertain Technol 2:27

    Google Scholar 

  23. Iso W (1998) 9241-11. Ergonomic requirements for office work with visual display terminals (VDTs). The international organization for standardization, 45, 9

  24. Järvinen A, Heliö S, Mäyrä F (2002) Communication and community in digital entertainment services. Prestudy Research Report

  25. Johnson D, Wiles J (2003) Effective affective user interface design in games. Ergonomics 46(13-14):1332–1345

    Article  Google Scholar 

  26. Ke F (2008) Alternative goal structures for computer game-based learning. Int J Comput-Support Collab Learn 3(4):429

    Article  Google Scholar 

  27. Ki King G, Krzywinska T (2002) Screenplay: cinema/videogames/interfaces. Wallflower Press

  28. Koeffel C, Hochleitner W, Leitner J, Haller M, Geven A, Tscheligi M (2010) Using heuristics to evaluate the overall user experience of video games and advanced interaction games. In Evaluating user experience in games (pp. 233-256). Springer, London

  29. Korhonen H (2010) Comparison of playtesting and expert review methods in mobile game evaluation. In Proceedings of the 3rd International Conference on Fun and Games (pp. 18-27). ACM

  30. Korhonen H (2011) The explanatory power of playability heuristics. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (p. 40). ACM

  31. Korhonen H, Koivisto EM (2006) Playability heuristics for mobile games. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services (pp. 9-16). ACM

  32. Korhonen H, Koivisto EM (2007) Playability heuristics for mobile multi-player games. In Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts (pp. 28-35). ACM

  33. Korhonen H, Paavilainen J, Saarenpää H (2009) Expert review method in game evaluations: comparison of two playability heuristic sets. In Proceedings of the 13th international MindTrek conference: Everyday life in the ubiquitous era (pp. 74-81). ACM

  34. Larsen JM (2008) Evaluating user experience–How game reviewers do it. In Evaluating User Experiences in Games, Workshop at CHI

  35. Law E, Roto V, Vermeeren AP, Kort J, Hassenzahl M (2008) Towards a shared definition of user experience In CHI'08 extended abstracts on Hum Factors in Comput Syst (pp. 2395-2398). ACM

  36. Lazar J, Feng JH, Hochheiser H (2017) Research methods in human-computer interaction. Morgan Kaufmann

  37. Livingston IJ, Mandryk RL, Stanley KG (2010) Critic-proofing: how using critic reviews and game genres can refine heuristic evaluations. In Proceedings of the International Academic Conference on the Future of Game Design and Technology (pp. 48-55). ACM

  38. Mack RL, Nielsen J (1994) Usability inspection methods. Wiley & Sons, New York, pp 1–414

    Google Scholar 

  39. Malone TW (1981) Toward a theory of intrinsically motivating instruction. Cogn Sci 5(4):333–369

    Article  Google Scholar 

  40. Malone TW (1982). Heuristics for designing enjoyable user interfaces: Lessons from computer games. In Proceedings of the 1982 conference on Hum Factors in Comput Syst(pp. 63-68). ACM

  41. Mandryk RL, Atkins MS (2007) A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int J Human-Comput Stud 65(4):329–347

    Article  Google Scholar 

  42. Myers D (2003) The Nature of Computer Games: Play As Semiosis (Digital Formations;, V. 16,). Peter Lang Publishing

  43. Nacke L (2009) From playability to a hierarchical game usability model. In Proceedings of the 2009 Conference on Future Play on@ GDC Canada (pp. 11-12). ACM

  44. Nielsen J (1994) Usability inspection methods. In Conference companion on Human factors in computing systems(pp. 413-414). ACM

  45. Nielsen J (1994) Usability engineering. Elsevier

  46. Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 249-256). ACM

  47. Papaloukas S, Patriarcheas K, Xenos M (2009). Usability assessment heuristics in new genre videogames. In Informatics, 2009. PCI'09. 13th Panhellenic Conference on (pp. 202-206). IEEE

  48. Pinelle, D., Wong, N., & Stach, T. (2008). Heuristic evaluation for games: usability principles for video game design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1453-1462). ACM

  49. Pinelle D, Wong N, Stach T, Gutwin C (2009) Usability heuristics for networked multiplayer games. In Proceedings of the ACM 2009 international conference on Supporting group work (pp. 169-178). ACM

  50. Röcker C, Haar M (2006) Exploring the usability of videogame heuristics for pervasive game development in smart home environments. In Proceedings of the third international workshop on pervasive gaming applications–pergames (pp. 199-206)

  51. Roll the Ball, https://play.google.com/store/apps/details?id=com.bitmango.rolltheballunrollme&hl=es_419 Retrieved in March 2018

  52. Sánchez JLG, Vela FLG, Simarro FM, Padilla-Zea N (2012) Playability: analysing user experience in video games. Behav Inform Technol 31(10):1033–1054

    Article  Google Scholar 

  53. Shadowmatic (2015) https://www.shadowmatic.com/ Triada Studio Games Retrieved in November 2017

  54. Susi T, Johannesson M, Backlund P (2007) Serious games: an overview. Technical Report HS­ IKI ­TR­07­001. School of Humanities and Informatics, University of Skövde, Sweden

  55. Sweetser P, Wyeth P (2005) GameFlow: a model for evaluating player enjoyment in games. Comput Entertain (CIE) 3(3):3–3

    Article  Google Scholar 

  56. Wolf MJ (2001) The medium of the video game. University of Texas Press

  57. Yáñez Gómez R, Cascado Caballero D, Sevillano JL (2014) Heuristic evaluation on mobile interfaces: A new checklist. Sci World J

  58. Yáñez-Gómez R, Cascado-Caballero D, Sevillano JL (2017) Academic methods for usability evaluation of serious games: a systematic review. Multimed Tools Appl 76(4):5755–5784

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the EU project SmokeFreeBrain (PI055-15/E03) and by the Telefonica Chair “Intelligence in Networks” of the Universidad de Sevilla, Spain. The licences for the Grim Fandango Remastered copies used during the tests where kindly provided by the creator of the remastered version and owner of its Intellectual Property, Double Fine Productions.

Author information

Authors and Affiliations

  1. Department of Computer Technology and Architecture, ETS Ingeniería Informática, Universidad de Sevilla, Avda Reina Mercedes s/n, 41012, Seville, Spain

    Rosa Yanez-Gomez, Juan Luis Font, Daniel Cascado-Caballero & José-Luis Sevillano

Authors
  1. Rosa Yanez-Gomez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Luis Font
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Cascado-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José-Luis Sevillano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rosa Yanez-Gomez.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yanez-Gomez, R., Font, J.L., Cascado-Caballero, D. et al. Heuristic usability evaluation on games: a modular approach. Multimed Tools Appl 78, 4937–4964 (2019). https://doi.org/10.1007/s11042-018-6593-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6593-1

Keywords

Search

Navigation