Advertisement

Serious Games pp 273-302 | Cite as

Performance Assessment in Serious Games

  • Josef Wiemeyer
  • Michael Kickmeier-Rust
  • Christina M. Steiner
Chapter

Abstract

In every digital game, players both act in and interact with the game. They use the options of game mechanics to achieve goals. For example, players move a controller to steer the motions of an avatar, or they press buttons to trigger certain actions. These actions and interactions lead to certain results like a successful finish of a quest, solving a problem, or increasing a score. Both the quality and results of actions and interactions are subsumed under the term performance. Assessment of player performance is required for several purposes, for example for in-game or online adaptation and for offline evaluation. This chapter addresses the issue of performance assessment in serious games. Performance is a complex concept comprising results and processes of actions and interactions of the players in and with the game. First, generic and domain-specific models of performance are introduced to illustrate the variety of approaches. Based on this knowledge, online and offline assessments of performance are discussed. Finally, the integration of online and offline performance assessment into the process of game adaptation is described.

Keywords

Human Performance Performance Assessment Intelligent Tutoring System Educational Game Berg Balance Scale 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Recommended Literature

  1. GALA (ed) (2014) Learning analytics for and in serious games. http://css-kmi.tugraz.at/mkrwww/leas-box/downloads/ectel14_booklet.pdf. Accessed 18 Feb 2016—This booklet gives an overview of current issues and methods in the field of learning analytics in serious games
  2. Ifenthaler D, Eseryel D, Ge X (eds) (2012) Assessment in game-based learning: Foundations, Innovations, and Perspectives. Springer, New York—This book provides an overview of assessment in games. Different assessment methods and methodologies are introduced and discussed concerning their specific strengths and weaknesses. Technological and methodological approaches are addressed, as well as issues of practical application of assessment in games and game-based learning Google Scholar
  3. Larusson JA & White B (eds) (2014) Learning Analytics—From Research to Practice. Springer, New York—This book is an introduction to learning analytics: Methods, theory, and practical applications are addressed Google Scholar

References

  1. 80Days (n.d.) 80Days—Around an inspiring virtual learning world in eighty days. www.eightydays.eu. Accessed 18 Feb 2016
  2. Abraham C, Michie S (2008) A taxonomy of behavior change techniques used in interventions. Health Psychol 27(3):379–387CrossRefGoogle Scholar
  3. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17:734–749CrossRefGoogle Scholar
  4. Ajzen I (1991) The theory of planned behaviour. Organ Behav Hum Dec 50:179–211Google Scholar
  5. Andersen KL, Shephard RJ, Denolin H, Varnauskas E, Masironi R (1971) Fundamentals of exercise testing. WHO, GenevaGoogle Scholar
  6. Baecke JA, Burema J, Frijters JE (1982) A short questionnaire for the measurement of habitual physical activity in epidemiological studies. Am J Clin Nutr 36(5):936–942Google Scholar
  7. Baker R, Inventado PS (2014) Educational data mining and learning analytics. In: Larusson JA, White B (eds) Learning analytics: from research to practice. Springer, New York, Germany, pp 61–75Google Scholar
  8. Baker R, Yacef K (2009) The state of educational data mining in 2009: a review and future visions. JEDM 1(1):3–17Google Scholar
  9. Baker RS, Habgood MJ, Ainsworth SE, Corbett AT (2007) Modeling the acquisition of fluent skill in educational action games. Proceedings of User Modeling 2007. Springer, Berlin, pp 17–26CrossRefGoogle Scholar
  10. Bakker I, Twisk JWR, Van Mechelen W, Mensink GBM, Kemper HCG (2003) Computerization of a dietary history interview in a running cohort: evaluation within the Amsterdam growth and health longitudinal study. Eur J Clin Nutr 57(3):394–404CrossRefGoogle Scholar
  11. Bandura A (1991) Social cognitive theory of self-regulation. Organ Behav Hum Dec 50 (2):248–287Google Scholar
  12. Baranowski T, Buday R, Thompson DI, Baranowski J (2008) Playing for real: video games and stories for health-related behavior change. Am J Prev Med 34(1):74–82CrossRefGoogle Scholar
  13. Bartlett R (2007) Introduction to sports biomechanics: analysing human movement patterns, 2nd edn. Routledge, LondonGoogle Scholar
  14. Becker MH, Haefner DP, Kasl SV, Kirscht JP, Maiman LA, Rosenstock IM (1977) Selected psychosocial models and correlates of individual health-related behaviors. Med Care 15(5):27–46CrossRefGoogle Scholar
  15. Bellotti F, Kapralos B, Lee K, Moreno-Ger P, Berta R (2013) Assessment in and of serious games: an overview. Adv Hum-Comput Interact 2013:136864. doi: 10.1155/2013/136864 Google Scholar
  16. Biddiss E, Irwin J (2010) Active video games to promote physical activity in children and youth: a systematic review. Arch Pediat Adol Med 164(7):664–672CrossRefGoogle Scholar
  17. Blum L, Korner-Bitensky N (2008) Usefulness of the berg balance scale in stroke rehabilitation: a systematic review. Phys Ther 88(5):559–566CrossRefGoogle Scholar
  18. Bös K (ed) (2001) Handbuch motorische tests. [Handbook of perceptual-motor tests] Hogrefe, GöttingenGoogle Scholar
  19. Bös K, Abel T, Woll A, Niemann S, Tittlbach S, Schott N (2002) Der Fragebogen zur Erfassung des motorischen Funktionsstatus (FFB-Mot). [Questionnaire for assessing the state of motor functions]. Diagnostica 48(2):101–111CrossRefGoogle Scholar
  20. Buschman TJ, Miller EK (2014) Goal-direction and top-down control. Philos Trans R Soc B 369(1655):0130471CrossRefGoogle Scholar
  21. Carvalho MB, Bellotti F, Berta R, De Gloria A, Gazzarata G, Hu J, Kickmeier-Rust M (2015) The journey: a service-based adaptive Serious Game on probability. Entertainment Comput 6:1–10CrossRefGoogle Scholar
  22. Chatti MA, Dyckhoff AL, Schroeder U, Thüs H (2012) A reference model for learning analytics. IJTEL 5:318–331CrossRefGoogle Scholar
  23. Deutsch JE, Guarrera-Bowlby P, Myslinski MJ, Kafri M (2015) Is there evidence that active videogames increase energy expenditure and exercise intensity for people poststroke and with cerebral palsy? Games Health J 4(1):31–36CrossRefGoogle Scholar
  24. Dickson-Spillmann M, Siegrist M, Keller C (2011) Development and validation of a short, consumer-oriented nutrition knowledge questionnaire. Appetite 56(3):617–620CrossRefGoogle Scholar
  25. Duval E (2011) Attention please! Learning analytics for visualization and recommendation. In: Proceedings of 1st international conference on learning analytics and knowledge. ACM, New York, pp 9–17Google Scholar
  26. Egenfeldt-Nielsen S (2006) Overview of research on the educational use of video games. Digital kompetanse 1(3):184–213Google Scholar
  27. EDM (2015) International data mining society. http://www.educationaldatamining.org/. Accessed 18 Feb 2016
  28. ELEKTRA (2014) ELECTRA—Enhancing learning in ENPI countries through clean technologies and research related activities. http://kti.tugraz.at/css/projects/elektra/. Accessed 18 Feb 2016
  29. Ferguson R (2012) Learning analytics: drivers, developments and challenges. IJTEL 4(5/6):304–317CrossRefGoogle Scholar
  30. Ferguson R, Shum SB (2012) Social learning analytics: five approaches. In: Proceedings of 2nd international conference on learning analytics and knowledge. ACM, New York, pp 23–33Google Scholar
  31. Finger J, Gößwald A, Härtel S, Müters S, Krug S, Hölling H, Kuhnert R, Bös K (2013) Measurement of cardiorespiratory fitness in the German health interview and examination survey for adults (DEGS1). Bundesgesundheitsbl 56:885–893CrossRefGoogle Scholar
  32. Fuchs R, Goehner W, Seelig H (2011) Long-term effects of a psychological group intervention on physical exercise and health: The MoVo concept. J Phys Act Health 8(6):794–803CrossRefGoogle Scholar
  33. Garcia AW, George TR, Coviak C, Antonakos C, Pender NJ (1997) Development of the child/adolescent activity log: a comprehensive and feasible measure of leisure-time physical activity. Int J Behav Med 4(4):323–338CrossRefGoogle Scholar
  34. Gladstone DJ, Danells CJ, Black SE (2002) The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehab Neural Re 16(3):232–240CrossRefGoogle Scholar
  35. Greller W, Drachsler H (2012) Translating learning into numbers: a generic framework for learning analytics. JETS 15:42–57Google Scholar
  36. Hay JG (1985) The biomechanics of sports techniques. Prentice Hall, New JerseyGoogle Scholar
  37. Heckhausen H (1991) Volition: Implementation of intentions. In: Heckhausen H (ed) Motivation and Action. Springer, Berlin, pp 163–188CrossRefGoogle Scholar
  38. Homer BD (2013) Overview. Introductory Talk to the Learning Analytics and Educational Data Mining Workshop,” CREATE Lab, New York University, April 2013 http://create.nyu.edu/wordpress/wp-content/uploads/2013/04/01.HOMER_.LEARNING.ANALYTICS.WORKSHOP.04.04.13.pdf. Accessed 18 Feb 2016
  39. Hope Labs (2015) Re-Mission. www.re-mission.net. Accessed 18 Feb 2016
  40. Houk JC, Wise SP (1995) Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5(2):95–110CrossRefGoogle Scholar
  41. Hurwitz J, Bloor R, Baroudi C, Kaufman M (2007) Service oriented architecture for dummies. Wiley, IndianapolisGoogle Scholar
  42. Ifenthaler D, Eseryel D, Ge X (eds) (2012b) Assessment in game-based learning: foundations, innovations, and perspectives. Springer, New YorkGoogle Scholar
  43. Kato PM, Cole SW, Bradlyn AS, Pollock BH (2008) A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics 122:e305–e317CrossRefGoogle Scholar
  44. Kearsley G (1993) Hypertext as a tool for the metatheoretical analysis of learning theories: the TIP database. JCHE 4(2):43–56Google Scholar
  45. Kickmeier-Rust MD, Albert D (2010) Micro adaptivity: protecting immersion in didactically adaptive digital educational games. J Comput Assist Lear 26:95–105CrossRefGoogle Scholar
  46. Kickmeier-Rust M, Albert D (2012a) Educationally adaptive: balancing Serious games. Int J Comput Sci Sport 11(1):15–28Google Scholar
  47. Kickmeier-Rust MD, Albert D (eds) (2012b) An Alien’s guide to multi-adaptive educational games. Informing Science Press, Santa Rosa, CAGoogle Scholar
  48. Kickmeier-Rust MD, Albert D (2013a) Gamification and intelligent feedback mechanisms for a division learning tool. In: Escudeiro P, de Carvalho CV (eds) Proceedings of 7th European conference on games based learning, Oct 2–3, 2013, Porto, Portugal, pp 290–296Google Scholar
  49. Kickmeier-Rust MD, Albert D (2013b) Learning analytics to support the use of virtual worlds in the classroom. In: Holzinger A, Pasi G (Eds) Human-computer interaction and knowledge discovery in complex, unstructured, big data. lecture notes in computer science, 7947, Springer, Berlin, pp 358–365Google Scholar
  50. Kivikangas JM, Chanel G, Cowley B, Ekman I, Salminen M, Järvelä S, Ravaja N (2011) A review of the use of psychophysiological methods in game research. JGVW 3(3):181–199CrossRefGoogle Scholar
  51. Knapik JJ, Jones BH, Reynolds KL, Staab JS (1992) Validity of self-assessed physical fitness. Am J Prev Med 8(6):367–372Google Scholar
  52. Koidl K, Mehm F, Hampson C, Conlan O, Göbel S (2010) Dynamically adjusting digital educational games towards learning objectives. In: Proceedings of 3rd European Conference on Game Based Learning (ECGBL), October 2010Google Scholar
  53. Krathwohl DR (2002) A revision of Bloom’s taxonomy: an overview. Theor Pract 41(4):212–218CrossRefGoogle Scholar
  54. Lager A, Bremberg S (2005) Health effects of video and computer game playing. A systematic review. Swedish National Institute of Public Health, StockholmGoogle Scholar
  55. Lee J (1999) Effectiveness of computer-based instructional simulation: a meta analysis. Int J Instr Media 26(1):71–85Google Scholar
  56. Lieberman DA (2001) Management of chronic pediatric diseases with interactive health games: theory and research findings. J Ambul Care Manage 24(1):26–38CrossRefGoogle Scholar
  57. Lockyer L, Dawson S (2011) Learning designs and learning analytics. In: Proceedings of 1st international conference on learning analytics and knowledge. ACM, New York, pp 153–156Google Scholar
  58. Mattheiss E, Kickmeier-Rust MD, Steiner CM, Albert D (2010) Approaches to detect discouraged learners: assessment of motivation in educational computer games. In: Proceedings of eLearning Baltics (eLBa) 2010, July 1–2, 2010, Rostock, GermanyGoogle Scholar
  59. Maud PJ, Foster C (2006) Physiological assessment of human fitness. Human Kinetics, Champaign, IllinoisGoogle Scholar
  60. McHorney CA, Ware JE Jr, Raczek AE (1993) The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 31(3):247–263CrossRefGoogle Scholar
  61. Meder N (2006) Web-Didaktik—Eine neue Didaktik webbasierten, vernetzten Lernens. [Web didactics—a new didactics of web-based networked learning] Bertelsmann, BielefeldGoogle Scholar
  62. Merrill MD (1983) Component display theory. In: Reigeluth CM (ed) Instructional-design theories and models: an overview of their current status. Erlbaum, Hillsdale, NJ, pp 279–333Google Scholar
  63. Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP (2011) A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol Health 26(11):1479–1498CrossRefGoogle Scholar
  64. Miller WL, Baker RS, Rossi LM (2014) Unifying computer-based assessment across conceptual instruction, problem-solving, and digital games. Technol Knowl Learn 19:165–181CrossRefGoogle Scholar
  65. Next Tell (2015) Next-Tell. Next generation teaching, education and learning for life. http://next-tell.de/. Accessed 18 Feb 2016
  66. Parmenter K, Wardle J (1999) Development of a general nutrition knowledge questionnaire for adults. Eur J Clin Nutr 53(4):298–308CrossRefGoogle Scholar
  67. Peng W, Lin J-H, Crouse JC (2011) Is playing exergames really exercising? A meta-analysis of energy expenditure in active video games. Cyberpsychol Behav Soc Netw 14(11):681–688CrossRefGoogle Scholar
  68. Peng W, Crouse JC, Lin JH (2013) Using active video games for physical activity promotion: a systematic review of the current state of research. Health Educ Behav 40(2):171–192CrossRefGoogle Scholar
  69. Poole DC, Wilkerson DP, Jones AM (2008) Validity of criteria for establishing maximal O2 uptake during ramp exercise tests. Eur J Appl Physiol 102(4):403–410CrossRefGoogle Scholar
  70. Prince SA, Adamo KB, Hamel ME, Hardt J, Gorber SC, Tremblay M (2008) A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act 5(1):56CrossRefGoogle Scholar
  71. Prochaska JO, Redding CA, Evers K (2008) The transtheoretical model and stages of change. In: Glanz K, Lewis FM, Rimer BK (eds) Health behavior and health education, 4th edn. Jossey-Bass, San Francisco, pp 97–121Google Scholar
  72. Reilly JJ, Penpraze V, Hislop J, Davies G, Grant S, Paton JY (2008) Objective measurement of physical activity and sedentary behaviour: review with new data. Arch Dis Child 93(7):614–619CrossRefGoogle Scholar
  73. Romero C (2010) Educational data mining: a review of the state of the art. IEEE Trans Syst Man Cybern C 40:601–618CrossRefGoogle Scholar
  74. Ryan RM, Deci EL (2000) Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 55(1):68–78CrossRefGoogle Scholar
  75. Schack T, Hackfort D (2007) Action theory approach to applied sport psychology. In: Tenenbaum G, Eklund RC (eds) Handbook of sport psychology, 3rd edn. John Wiley and Sons, New York, pp 332–351Google Scholar
  76. Schwarzer R (2008) Modeling health behavior change: how to predict and modify the adoption and maintenance of health behaviors. Appl Psychol 57(1):1–29CrossRefGoogle Scholar
  77. Serious Games Society (n.d.). Serious games web services catalog. http://services.seriousgamessociety.org/. Accessed 18 Feb 2016
  78. Serrano-Laguna A, Torrente J, Moreno-Ger P, Fernández-Manjón B (2012) Tracing a little for big improvements: application of learning analytics and videogames for student assessment. Procedia Comput Sci 15:203–209CrossRefGoogle Scholar
  79. Shephard RJ (2003) Limits to the measurement of habitual physical activity by questionnaires. Br J Sports Med 37(3):197–206MathSciNetCrossRefGoogle Scholar
  80. Siemens G (2012) Learning analytics: envisioning a research discipline and a domain of practice. In: Proceedings of international conference on learning analytics and knowledge, ACM, New York, pp 4–8Google Scholar
  81. Staiano AE, Flynn R (2014) Therapeutic uses of active videogames: a systematic review. Games Health J 3(6):351–365CrossRefGoogle Scholar
  82. Stephenson S, Baker R, Corrigan S (2014) Towards building an automated detector of engaged and disengaged behavior in game-based assessments. In: Poster, annual conference on games + learning + societyGoogle Scholar
  83. Strøyer J, Essendrop M, Jensen LD, Warming S, Avlund K, Schibye B (2007) Validity and reliability of self-assessed physical fitness using visual analogue scales. Percept Mot Skills 104(2):519–533CrossRefGoogle Scholar
  84. Sween J, Wallington SF, Sheppard V, Taylor T, Llanos AA, Adams-Campbell LL (2014) The role of exergaming in improving physical activity: a review. J Phys Act Health 11(4):864–870CrossRefGoogle Scholar
  85. Tenenbaum G, Eklund RC (eds) (2007) Handbook of sport psychology. Wiley, New York, NYGoogle Scholar
  86. Vogel JJ, Vogel DS, Cannon-Bowers J, Bowers JC, Muse K, Wright M (2006) Computer gaming and interactive simulations for learning: a meta-analysis. J Educ Comput Res 34(3):229–243CrossRefGoogle Scholar
  87. Wagner P (2000) Aussteigen oder Dabeibleiben? [Drop out or continue?] Wissenschaftliche Buchgesellschaft, DarmstadtGoogle Scholar
  88. Ware JE Jr (2000) SF-36 health survey update. Spine 25(24):3130–3139CrossRefGoogle Scholar
  89. Ware JE Jr, Sherbourne CD (1992) The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care 30(6):473–483CrossRefGoogle Scholar
  90. Wiemeyer J (2003) Function as constitutive feature of movements in sport. Int J Comput Sci Sport 2(2):113–115Google Scholar
  91. Wiemeyer J (2014) Serious games in neurorehabilitation—a review of recent evidence. SeriousGames’14, Proc. ACM international workshop on serious games. ACM, New York, pp 33–38Google Scholar
  92. Wiemeyer J, Hardy S (2013) Serious games and motor learning—concepts, evidence, technology. In: Bredl K, Bösche W (eds) Serious games and virtual worlds in education, professional development and healthcare. IGI Global, Heshey, PA, pp 197–220CrossRefGoogle Scholar
  93. Williams SL, French DP (2011) What are the most effective intervention techniques for changing physical activity self-efficacy and physical activity behaviour—and are they the same? Health Educ Res 26(2):308–322CrossRefGoogle Scholar
  94. Wilson AM, Magarey AM, Mastersson N (2008) Reliability and relative validity of a child nutrition questionnaire to simultaneously assess dietary patterns associated with positive energy balance and food behaviours, attitudes, knowledge and environments associated with healthy eating. Int J Behav Nutr Phys Act 5(1):5CrossRefGoogle Scholar
  95. World Health Organization [WHO] (1992) The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. World Health Organization, GenevaGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Josef Wiemeyer
    • 1
  • Michael Kickmeier-Rust
    • 2
  • Christina M. Steiner
    • 2
  1. 1.Technische Universität DarmstadtDarmstadtGermany
  2. 2.Graz University of TechnologyGrazAustria

Personalised recommendations