Extending Empirical Analysis of Usability and Playability to Multimodal Computer Games
The published research examining usability and playability of games is largely theoretical. A prior empirical study of a game with an embodied conversational agent found that most frustration episodes could be understood in terms of both usability and playability, but this study was based on a game in which the interaction by both player and agent were limited to verbal communication. To explore whether these results would hold for a game in which the player and agent communicated with both speech and gesture, we conducted an empirical formative user-experience evaluation of a multimodal game. Our findings strongly confirmed that frustration episodes can be understood as issues of both usability and playability. However, the relative frequencies of the categories of usability and playability issues differed between the speech-only and the speech-and-gesture games. Much of this difference likely arose because higher levels of engagement and rapport between player and agent in the speech-and-gesture game led to the players having greater, and in many cases unfulfilled, expectations for the capabilities of the agent.
KeywordsEmbodied conversational agent User experience Playability Usability Gesture Engagement Rapport
The authors acknowledge with gratitude the team, in addition to the authors, that developed Survival on Jungle Island: Ivan Gris, Adriana Camacho, Diego Rivera, Mario Gutierrez, Alex Rayon, Joel Quintana, Anuar Jauregui, Timothy Gonzales, Alfonso Peralta, Victoria Bravo, Jacqueline Brixey, Yahaira Reyes, Paola Gallardo, Chelsey Jurado, Guillaume Adoneth, David Manuel, Brynne Blaugrund, and Nick Farber.
- 1.Sánchez, J.G., Simarro, F.M., Zea, N.P., Vela, F.G.: Playability as extension of quality in use in video games. In: 2nd International Workshop on the Interplay between Usability Evaluation and Software Development (I-USED) (2009)Google Scholar
- 3.Cassell, J.: Embodied Conversational Agents. MIT press, Cambridge (2000)Google Scholar
- 4.Novick, D., Vicario, J., Santaella, B., Gris, I.: Empirical analysis of playability vs. usability in a computer game. In: Marcus, A. (ed.) DUXU 2014, Part II. LNCS, vol. 8518, pp. 720–731. Springer, Heidelberg (2014)Google Scholar
- 5.Desurvire, H., Caplan, M., Toth, J.A.: Using heuristics to evaluate the playability of games. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1509–1512. ACM (2004)Google Scholar
- 6.Pinelle, D., Wong, N., Stach, T.: Heuristic evaluation for games: usability principles for video game design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1453–1462. ACM (2008)Google Scholar
- 7.Fierley, R., Engl, S.: User experience methods and games: lessons learned. In: proceedings of the 24th BCS Interaction Specialist Group Conference, pp. 204–210. British Computer Society, Swinton (2010)Google Scholar
- 8.Nacke, L.: From playability to a hierarchical game usability model. In: 2009 Conference on Future Play on@ GDC Canada, pp. 11–12 (2009)Google Scholar
- 12.Novick, D., Gris, I., Rivera, D.A., Camacho, A., Rayon, A., Gutierrez, M.: The UTEP AGENT System. In: Proceedings of the 17th ACM International Conference on Multimodal Interaction, Seattle, WA, 9–13 November 2015Google Scholar
- 13.Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256 (1990)Google Scholar
- 14.University Communications [of the Univ. Texas at El Paso]: UTEP Computer Science Department develops award-winning interactive agent system, 21 January 2016. http://engineering.utep.edu/announcement012116.htm