Challenges in the Development and Evaluation of Immersive Digital Educational Games

  • Effie Lai-Chong Law
  • Michael D. Kickmeier-Rust
  • Dietrich Albert
  • Andreas Holzinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5298)


In this paper, we describe a conceptual framework and address the related issues and solutions in the identification of three major challenges for the development and evaluation of Immersive Digital Educational Games (IDEGs). These challenges are (i) advancing adaptive educational technologies to shape learning experience, ensuring the individualization of learning experiences, adaptation to personal aims, needs, abilities and prerequisites; (ii) providing technological approaches to reduce the development costs for IDEGs; by enabling the creation of entirely different stories and games for a variety of different learning domains, each based, more or less, on the same pool of story units; patterns and structures; (iii) developing robust evaluation methodologies for IDEGs by the extension of ISO 9241 to include user satisfaction, motivation and learning progress and other User Experience (UX) attributes. While our research and development is by no means concluded, we believe that we have arrived at a stage where conclusions may be drawn, which will be of considerable use to other researchers in this domain.


Immersive digital educational games Marco adaptivity Micro adaptivity User experience Reusability User Experience 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Albert, D., Lukas, J.: Knowledge spaces: theories, empirical research, and applications. Lawrence Erlbaum Associates, Mahwah (1999)Google Scholar
  2. 2.
    Holzinger, A., Pichler, A., Almer, W., Maurer, H.: TRIANGLE: A Multi-Media test-bed for examining incidental learning, motivation and the Tamagotchi-Effect within a Game-Show like Computer Based Learning Module Educational Multimedia, Hypermedia and Telecommunication 2001, Association for the Advancement of Computing in Education, Charlottesville (VA), pp. 766–771 (2001)Google Scholar
  3. 3.
    Mann, B.D., Eidelson, B.M., Fukuchi, S.G., Nissman, S.A., Robertson, S., Jardines, L.: The development of an interactive game-based tool for learning surgical management algorithms via computer. The American Journal of Surgery 183(3), 305–308 (2002)CrossRefGoogle Scholar
  4. 4.
    Cai, Y., Snel, I., Bharathi, B.S., Klein, C., Klein-Seetharaman, J.: Towards biomedical problem solving in a game environment. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J., Zomaya, A.Y. (eds.) ICCS 2003. LNCS, vol. 2659, pp. 1005–1014. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Holzinger, A., Pichler, A., Maurer, H.: Multi Media e-Learning Software TRIANGLE Case-Study: Experimental Results and Lessons Learned. Journal of Universal Science and Technology of Learning 0(0), 61–92 (2006), Google Scholar
  6. 6.
    de Freitas, S., Oliver, M.: How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & Education 46(3), 249–264 (2006)CrossRefGoogle Scholar
  7. 7.
    Ebner, M., Holzinger, A.: Successful Implementation of User-Centered Game Based Learning in Higher Education – an Example from Civil Engineering. Computers & Education 49(3), 873–890 (2007)CrossRefGoogle Scholar
  8. 8.
    Robertson, J., Howells, C.: Computer game design: Opportunities for successful learning. Computers & Education 50(2), 559–578 (2008)CrossRefGoogle Scholar
  9. 9.
    Garris, R., Ahlers, R., Driskell, J.E.: Games, Motivation, and Learning: A Research and Practice Model. Simulation & Gaming 33(4), 441–467 (2002)CrossRefGoogle Scholar
  10. 10.
    Prensky, M.: Digital game-based Learning. McGraw Hill, New York (2001)Google Scholar
  11. 11.
    Merrill, M.D.: First principles of instruction. Etr&D-Educational Technology Research and Development 50(3), 43–59 (2002)CrossRefGoogle Scholar
  12. 12.
    Cordova, D.I., Lepper, M.R.: Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology 88(4), 715–730 (1996)CrossRefGoogle Scholar
  13. 13.
    Holzinger, A.: Computer-aided Mathematics Instruction with Mathematica 3.0. Mathematica in Education and Research 6(4), 37–40 (1997)Google Scholar
  14. 14.
    Shneiderman, B.: Relate-Create-Donate: a teaching/learning philosophy for the cyber-generation. Computers & Education 31(1), 25–39 (1998)CrossRefGoogle Scholar
  15. 15.
    van Reekum, C.M., Johnstone, T., Banse, R., Etter, A., Wehrle, T., Scherer, K.R.: Psychophysiological responses to appraisal dimensions in a computer game. Cognition & Emotion 18(5), 663–688 (2004)CrossRefGoogle Scholar
  16. 16.
    ELEKTRA: ELEKTRA - Enhanced Learning Experience and Knowledge TRAnsfer (last access: 2008-09-06),
  17. 17.
    Aviram, R., Winer, A., Somech, S., Schellas, Y., Dotan, I., Ronen, Y.: iClass Methodologies: Intelligent Distributed Cognitive-based Open Learning System for Schools (last access: 2008-09-06),
  18. 18.
    Wouters, P., Tabbers, H.K., Paas, F.: Interactivity in video-based models. Educational Psychology Review 19(3), 327–342 (2007)CrossRefGoogle Scholar
  19. 19.
    Entwistle, N., McCune, V.: The conceptual bases of study strategy inventories. Educational Psychology Review 16(4), 325–345 (2004)CrossRefGoogle Scholar
  20. 20.
    Caprara, G.V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G.M., Barbaranelli, C., Bandura, A.: Longitudinal analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology 100(3), 525–534 (2008)CrossRefGoogle Scholar
  21. 21.
    Boekaerts, M.: Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and Instruction 7(2), 161–186 (1997)CrossRefGoogle Scholar
  22. 22.
    Holzinger, A., Kickmeier-Rust, M., Albert, D.: Dynamic Media in Computer Science Education; Content Complexity and Learning Performance: Is Less More? Educational Technology & Society 11(1), 279–290 (2008)Google Scholar
  23. 23.
    Kickmeier-Rust, M., Albert, D.: The ELEKTRA ontology model: A learner-centered approach to resource description. LNCS. Springer, Heidelberg (in press)Google Scholar
  24. 24.
    Holzinger, A., Nischelwitzer, A.: Chameleon Learning Objects: Quasi-Intelligente doppelt adaptierende Lernobjekte: Vom Technologiemodell zum Lernmodell. OCG Journal 30(4), 4–6 (2005)Google Scholar
  25. 25.
    Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behaviour & Information Technology 25(2), 91–97 (2006)CrossRefGoogle Scholar
  26. 26.
    Law, E., Hvannberg, E.T., Hassenzahl, M.: Proceedings of the Workshop Towards a Unified View of UX (last access: 2008-09-10),
  27. 27.
    van Schaik, P., Ling, J.: Modelling user experience with web sites: Usability, hedonic value, beauty and goodness. Interacting with Computers 20(3), 419–432 (2008)CrossRefGoogle Scholar
  28. 28.
    Hassenzahl, M., Beu, A., Burmester, M.: Engineering joy. IEEE Software 18(1), 70 (2001)CrossRefGoogle Scholar
  29. 29.
    Harbich, S., Auer, S.: Rater bias: The influence of hedonic quality on usability questionnaires. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1129–1133. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  30. 30.
    Schrepp, M., Held, T., Laugwitz, B.: The influence of hedonic quality on the attractiveness of user interfaces of business management software. Interacting with Computers 18(5), 1055–1069 (2006)CrossRefGoogle Scholar
  31. 31.
    Hassenzahl, M., Platz, A., Burmester, M., Lehner, K.: Hedonic and ergonomic quality aspects determine a software’s appeal. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 201–208 (2000)Google Scholar
  32. 32.
    Wideman, H.H., Owston, R.D., Brown, C., Kushniruk, A., Ho, F., Pitts, K.C.: Unpacking the potential of educational gaming. Simulation & Gaming 38(1), 10–30 (2007)CrossRefGoogle Scholar
  33. 33.
    Holzinger, A.: Multimedia Basics, Laxmi, New Delhi. Learning. Cognitive Fundamentals of multimedial Information Systems, vol. 2 (2002),
  34. 34.
    Holzinger, A.: Usability Engineering for Software Developers. Communications of the ACM 48(1), 71–74 (2005)CrossRefGoogle Scholar
  35. 35.
    Stephanie, R., Janice Anne, R., Judee, H.: A toolkit for strategic usability: results from workshops, panels, and surveys. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 337–344 (2000)Google Scholar
  36. 36.
    Holzinger, A.: Application of Rapid Prototyping to the User Interface Development for a Virtual Medical Campus. IEEE Software 21(1), 92–99 (2004)CrossRefGoogle Scholar
  37. 37.
    Bernhaupt, R., Eckschlager, M., Tscheligi, M.: Methods for evaluating games: how to measure usability and user experience in games? In: Proceedings of the international conference on Advances in computer entertainment technology (2007)Google Scholar
  38. 38.
    Fabricatore, C., Nussbaum, M., Rosas, R.: Playability in action videogames: A qualitative design model. Human-Computer Interaction 17(4), 311–368 (2002)CrossRefGoogle Scholar
  39. 39.
    Desurvire, H., Caplan, M., Toth, J.A.: Using heuristics to evaluate the playability of games. In: CHI 2004, pp. 1509–1512 (2004)Google Scholar
  40. 40.
    Korhonen, H., Koivisto, E.M.I.: Playability Heuristics for Mobile Multi-Player Games. In: DIMEA 2007, Second International Conference on Digital Interactive Media in Entertainment and Arts (2007)Google Scholar
  41. 41.
    Röcker, C., Haar, M.: Exploring the usability of videogame heuristics for pervasive game development in smart home enviroments. In: Third International Workshop on Pervasive Game Applications - PerGames 2006, pp. 199–206 (2006)Google Scholar
  42. 42.
    Mandryk, R.L., Atkins, M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies 65(4), 329–347 (2007)CrossRefGoogle Scholar
  43. 43.
    van den Haak, M., De Jong, M., Schellens, P.J.: Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behaviour & Information Technology 22(5), 339–351 (2003)CrossRefGoogle Scholar
  44. 44.
    Van den Haak, M.J., de Jong, M.D.T., Schellens, P.J.: Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Interacting with Computers 16(6), 1153–1170 (2004)CrossRefGoogle Scholar
  45. 45.
    Csíkszentmihályi, M.: Finding flow: The Psychology of engagement with everyday life. Basic Books, London (1998)Google Scholar
  46. 46.
    de Lera, E., Garreta-Domingo, M.: Ten emotion heuristics: Guidelines for assessing the user’s affective dimension easily and cost effectively. In: Proceedings of HCI 2007 The 21st British HCI Group Annual Conference, University of Lancaster, September 3 - 7 (2007)Google Scholar
  47. 47.
    Stickel, C., Fink, J., Holzinger, A.: Enhancing Universal Access – EEG based Learnability Assessment. In: Stephanidis, C. (ed.) HCI 2007. LNCS, vol. 4556, pp. 813–822. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  48. 48.
    Stickel, C., Holzinger, A., Ebner, M.: Useful Oblivion Versus Information Overload in e-Learning Examples in the Context of Wiki Systems. In: Luzar-Stiffler, V., Dobric, V.H., Bekic, Z. (eds.) Proceedings of the ITI 2008 30th International Conference on Information Technology Interface, June 23-26, pp. 171–176 (2008)Google Scholar
  49. 49.
    Bernhaupt, R., IJsselsteij, W., Mueller, F., Tscheligi, M., Wixon, D.R.: Evaluating user experience games CHI 2008 extended abstracts on Human factors in computing systems, pp. 3905–3908 (2008)Google Scholar
  50. 50.
    Lockee, B., Moore, M., Burton, J.: Measuring success: Evaluation strategies for distance education. Educause Quaterly 25(1), 20–26 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Effie Lai-Chong Law
    • 1
  • Michael D. Kickmeier-Rust
    • 2
  • Dietrich Albert
    • 2
  • Andreas Holzinger
    • 3
  1. 1.Department of Computer ScienceUniversity of LeicesterLeicesterUK
  2. 2.Department of Psychology Cognitive Science SectionGraz UniversityGrazAustria
  3. 3.Institute for Medical Informatics, Statistics & Documentation (IMI) Research Unit HCI4MEDMedical University GrazGrazAustria

Personalised recommendations