Advertisement

Transfer of Training of An Educational Serious Game: The Effectiveness of the CASHIER TRAINER

  • E. A. P. B (Esther) Oprins
  • J. E (Hans) Korteling
Chapter
Part of the Gaming Media and Social Effects book series (GMSE)

Abstract

We investigated the transfer of training of a stand-alone educational serious game supported with automated feedback and instruction compared to conventional on-the-job training (OJT). If transfer of training is sufficiently high, these types of games could reduce time-consuming and expensive OJT saving instructional personnel. In a case study with such a game, the CASHIER TRAINER supported with an ITS, we compared performance and competence of new cashier employees at the workplace in two matched groups: (a) experimental group: acquiring cashier skills in the game CASHIER TRAINER; (b) control group: acquiring cashier skills in conventional OJT. Performance and competence were measured by observation and by self-assessment. The results showed that performance and competence were at least as high and even higher on certain aspects for the CASHIER TRAINER than for the OJT condition. Thus, transfer of training with the CASHIER TRAINER was positive in comparison with OJT. Specifically, non-regular tasks which could be trained with gaming more often and structured than in OJT were performed better. It seems that automated feedback in stand-alone educational serious games for procedural tasks, if well-designed, could replace human tutoring to a certain extent.

Keywords

Evaluation Educational games  Instructional games Serious games Intelligent tutoring Training effectiveness  Instructional effectiveness Training simulation Transfer of trainingTransfer of training 

Notes

Acknowledgments

This study was a Knowledge Transfer Project of the research program Games for Training and Entertainment (GATE) which was supported by the Dutch ICT-Regie and NWO. We would like to thank Jutten Simulation for using the game CASHIER TRAINER as a case in this study. We also would like to thank the retail company Sligro Foodgroup n.v. in The Netherlands for making it possible to collect data in their EMTÉ supermarkets.

References

  1. Akl EA, Pretorius RW, Sackett K, Scott Erdley W, Bhoopath P, Alfarah Z, Nemann HJ (2010) The effect of educational games on medical students’ learning outcomes: a systematic review. BEME guide no 14Google Scholar
  2. Allan JA, Hays JT, Buffardi LC (1986) Maintenance Training Simulator Fidelity and Individual Differences in Transfer of Training. Hum Factors 28:297–509Google Scholar
  3. Alklind Taylor AS, Backlund P, Niklasson L (2012) The Coaching Cycle: A Coaching-by-Gaming Approach in Serious Games. Simul Gaming 20(10):1–25Google Scholar
  4. Baldwin TT, Ford JK (1988) Transfer of training: a review and directions for future research. Pers Psychol 41:63–105CrossRefGoogle Scholar
  5. Bandura A (1997) Self-efficacy: the exercise of control. Freeman, New YorkGoogle Scholar
  6. Bedwell WL, Pavlas D, Heyne K, Lazzara EH, Salas E (2012) Toward a taxonomy linking game attributes to learning: an empirical study. Simul Gaming 43(6):729–760CrossRefGoogle Scholar
  7. Bekebrede G, Warmelink HJG, Mayer IS (2011) Reviewing the need for gaming in education to accommodate the net generation. Comput Educ 57(2):1521–1529. doi: 10.1016/j.compedu.2011.02.010 CrossRefGoogle Scholar
  8. Bell BS, Kanar AM, Kozlowski SWJ (2008) Current issues and future directions in simulation-based training in North America. Int J Hum Res Manag 19:1416–1436CrossRefGoogle Scholar
  9. Caro, P.W. (1977). Some factors influencing air force simulator training effectiveness HUMRRO Technical Report tr-77–2. Alexandria, Virginia: Human Resources Research OrganisationGoogle Scholar
  10. Cohn J, Kay S, Milham L, Bell Carroll M, Jones D, Sullivan J, Darken R (2009) Training effectiveness evaluation: from theory to practice. In: Schmorrow D, Cohn J, Nicholson D (eds). The PSI handbook of virtual environments for training and education,pp 157–172Google Scholar
  11. Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM (2012) A systematic literature review of empirical evidence on computer games and serious games. Comput Educ 59(2):661–686. doi: 10.1016/j.compedu.2012.03.004 CrossRefGoogle Scholar
  12. Crookall D (2010) Serious games, debriefing, and simulation/gaming as a discipline. Simul Gaming 41(6):898–920CrossRefGoogle Scholar
  13. Csikszentmihali M (1990) Flow: the psychology of optimal experience. Harper and Row, New YorkGoogle Scholar
  14. Egenfeldt-Nielsen S (2006) Overview of research on the educational use of video games. Digital Kompetanse 1(3):184–213Google Scholar
  15. Emmerik ML, van, Korteling JE (2002) Certificering van trainingssimulatoren 2: de TNO-TM checklist. [Certification of training simulators 2: The TNO-TM checklist] Report TM-02-D010. Soesterberg, The Netherlands: TNO Human Factors research InstituteGoogle Scholar
  16. Emmerik ML, van, Korteling JE (2003) Certificering van trainingssimulatoren 3: Computer-gebaseerd ONdersteuningmiddel voor CERtificering van Trainingssimulatoren. [Certification of training simulators 3: Computer-based Support Tool for Certification of Training Simulators]  Report TM-03-D005. Soesterberg, The Netherlands: TNO Human Factors research InstituteGoogle Scholar
  17. Farmer E, van Rooij J, Riemersma J, Jorna P, Moraal J (1999) Handbook of simulator-based training. Ashgate, AldershotGoogle Scholar
  18. Fletcher JD, Tobias S (2008) What research has to say (thus far) about designing computer games for learning. Paper presented at the American Educational Research Association, New YorkGoogle Scholar
  19. Fowlkes J E, Dwyer DJ, Milham L M, Burns J J, Pierce L G (1999) Team skills assessment: A test and evaluation component for emerging weapon systems. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FLGoogle Scholar
  20. Freitas S, de (2006) Learning in immersive worlds: a review of game-based learning (Tech. Rep.). Joint Information Systems Committee Bristol, BristolGoogle Scholar
  21. Gee JP (2007) What videogames have to teach us about learning and literacy. Palgrave Macmillan, New YorkGoogle Scholar
  22. Gielen EWM (1995) Transfer of training in a corporate setting (doctoral thesis). University Twente, EnschedeGoogle Scholar
  23. Girard C, Ecalle J, Magnan A (2012) Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. J Computer Assist Learn. doi: 10.1111/j.1365-2729.2012.00489.x Google Scholar
  24. Harteveld C (2011) Triadic game design: Balancing reality, meaning and play. Springer, LondonCrossRefGoogle Scholar
  25. Harteveld C (2012) Making sense of Virtual Risks: a Quasi-Experimental Investigation into Game-Based Training. IOS Press, AmsterdamGoogle Scholar
  26. Harteveld C, Bekebrede G (2011) Learning in Single-Versus Multiplayer Games: The More the Merrier? Simul Gaming 42(1):43–63CrossRefGoogle Scholar
  27. Hays RT (2005) The effectiveness of instructional games: a literature review and discussion. Technical Report 2005-004. Naval Air Warfare Training Systems Division. Orlando, U.S.AGoogle Scholar
  28. Jiusto S, DiBiasio D (2006) Experiential learning environments: Do they prepare our students to be self-directed, life-long learners? J Eng Educ 95:195–204CrossRefGoogle Scholar
  29. Johnston S, McCormack C (1996) Integrating information technology into university teaching: Identifying the needs and providing the support. Int J Educ Manag 10(5):36–42CrossRefGoogle Scholar
  30. Ke F (2009) A qualitative meta-analysis of computer games as learning tools. In Ferdig RE (ed) Handbook of research on effective electronic gaming in education vol I. pp 1–32Google Scholar
  31. Kirkpatrick DI (1998) Evaluating Training Programs: The Four Levels, 2nd edn. Berrett-Koehler, San FranciscoGoogle Scholar
  32. Korteling JE, Helsdingen AS, Theunissen NCM (2012) Serious Games @ Work Learning job-related competences using serious gaming. In: Bakker A, Derks D (eds) The Psychology of Digital Media at Work. London, New York: Psychology Press LTD/Taylor and Francis Group Google Scholar
  33. Korteling JE (2012) Evaluation of a cashier trainer: true evidence for transfer of training of serious gaming. In: Veltkamp R (ed) Growing knowledge for games utrecht NL: Control Magazine 68–69Google Scholar
  34. Korteling JE, Oprins EAPB, Kallen VL (2013) Measurement of Effectiveness for training simulations. In: Wang Z (ed) RTO-MP-SAS-095 Cost-Benefit Analysis of Military Training. Paper presented at the SAS Workshop held in Amsterdam, The Netherlands, 5–6 June 2012. [http://www.cso.nato.int/abstracts.aspx]
  35. Korteling JE, Padmos P, Helsdingen AS, Sluimer RR (2001) Certificering van trainingssimulatoren 1: kennisinventarisatie [Certification of training simulators 1: knowledge inventarization] Report TM-01-D003. TNO Human Factors Research Institute, Soesterberg, The NetherlandsGoogle Scholar
  36. Leemkuil H, de Jong T, Ootes S (2000) Review of educational use of games and simulations (IST-1999-13078 Deliverable D1). University of Twente, EnschedeGoogle Scholar
  37. Lee J (1999) Effectiveness of a computer-based instructional simulation: a meta-analysis. Int J Instr Media 26:71–85Google Scholar
  38. Malone TW (1981) Towards a theory of intrinsically motivating instruction. Cognitive Sci 4:333–369CrossRefGoogle Scholar
  39. Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60MathSciNetCrossRefMATHGoogle Scholar
  40. van Merrienboer JJG, Kirschner P (2007) Ten steps to complex learning: a systematic approach to four-component instructional design. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  41. O’Neil HF, Wainess R, Baker EL (2005) Classification of learning outcomes: evidence from the computer games literature. Curric J 16(4):455–474. doi: 10.1080/09585170500384529 CrossRefGoogle Scholar
  42. Oprins E (2008) Design of a competence-based assessment system for air traffic control training. Maastricht University, Doctoral dissertationGoogle Scholar
  43. Oprins E, Burggraaff E, Roe R (2011) Analysis of learning curves in the on-the-job training of air traffic controllers. In: D’Oliviera TC (ed) Mechanisms in the chain of safety. Ashgate Publishing Company, AldershotGoogle Scholar
  44. Oprins E, Burggraaff E, Van Weerdenburg H (2006) Design of a competence-based assessment system for air traffic control training. Int J Aviat Psychol 16(3):297–320CrossRefGoogle Scholar
  45. Oprins E, Burggraaff E, Van Weerdenburg H (2008) Reliability of assessors’ ratings in competence-based air traffic control training. Hum Factors Aerosp Saf 6(4):305–322Google Scholar
  46. Peck M (2012) Tools or toys? Training games are popular, but no one knows how well they work. Train simul J dec2011/jan2012 Google Scholar
  47. Percival A (1996) Invited reaction: An adult educator responds. Hum Resour Dev Quart 7:131–139CrossRefGoogle Scholar
  48. Petraglia J (1998) Reality by design: the rhetoric and technology of authenticity in education. Erlbaum, MahwahGoogle Scholar
  49. Phillips DC (1998) How, why, what, when, and where: Perspectives on constructivism in psychology and education. Issues Educ 3:151–194Google Scholar
  50. Polson M, Richardson J (eds) (1988) Foundations of Intelligent Tutoring Systems. Lawrence Erlbaum Associates, HillsdaleGoogle Scholar
  51. Randal JM, Morris BA, Wetzel CD, Whitehill BV (1992) The effectiveness of games for educational purposes: a review of recent research. Simul Gaming 23:261–276CrossRefGoogle Scholar
  52. Roscoe SN, Williges BH (1980) Measurement of transfer of training. In: Roscoe SN (ed) Aviation Psychology. The Iowa State University Press, IowaGoogle Scholar
  53. Salas E, Milham LM, Bowers CA (2003) Training evaluation in the military: misconceptions, opportunities, and challenges. Mil Psychol 15:3–16CrossRefGoogle Scholar
  54. Salas E, Rosen MA, Held JD, Weissmuller JJ (2009) Performance Measurement in Simulation-Based Training: A Review and Best Practices. Simul Gaming 40(3):328–376CrossRefGoogle Scholar
  55. Sitzmann T (2011) A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Pers Psychol 64:489–528CrossRefGoogle Scholar
  56. Smith PAC, O’Neil J (2003) A review of action learning literature 1994-2000. Part 1: Bibliography and comments. J Workplace Learn 15:63–69CrossRefGoogle Scholar
  57. Stainton AJ, Johnson JE, Borodzicz EP (2010) Educational Validity of Business Gaming Simulation: a research methology framework. Simul Gaming 41(5):705–723CrossRefGoogle Scholar
  58. Steffe L, Gale J (eds) (1995) Constructivism in education. Lawrence Erlbaum Associates, Inc., HillsdaleGoogle Scholar
  59. Stehouwer M, Serné M, Niekel C (2005) A tactical trainer for air defence platoon commanders. In: Proceedings of the interservice/industry, training, simulation, and education conference. Orlando I/ITSEC 2005, Paper no. 206Google Scholar
  60. Shaffer (2006) How computer games help people learn. Palgrave Macmillan, New YorkGoogle Scholar
  61. Squire K (2003) Video games in education. Int J Intell Simul Gaming 2(1):49–62Google Scholar
  62. Stubbé HM, Theunissen NCM (2008) Self-directed learning in a ubiquitous learning environment: a meta-review. In: Proceedings of special track on technology support for self-organised learners 2008, pp 5–28Google Scholar
  63. Thorndike EL (1920) A constant error in psychological ratings. J Appl Psychol 4:25–29CrossRefGoogle Scholar
  64. Tobias S, Fletcher JD (2007) What research has to say about designing computer games for learning. Educ Technol 47:20–29Google Scholar
  65. Veldhuis GJ, Theunissen NCM (2009) Transfer of Training. Onderzoek naar een maximaal leereffect [Transfer of training. Research on a maximal learning effect]. Opleiding en Train 11:20–22Google Scholar
  66. Vogel J, Vogel DS, Cannon-Bowers J, Bowers CA, Muse K, Wright M (2006) Computer gaming and interactive simulations for learning: a meta-analysis. J Edu Comput Res 34:229–243CrossRefGoogle Scholar
  67. Wouters P, van der Spek E, Van Oostendorp H (2009) Current practices in serious game research: a review from a learning outcomes perspective. Games-based learning advancements for multi-sensory human computer interfaces: techniques and effective practices 232–250Google Scholar
  68. Young MF, Slota S, Cutter AB, Yukhymenko M (2012) Our princess is in another castle: A review of trends in serious gaming for education. Rev Educ Res 82(1):61–89Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2014

Authors and Affiliations

  • E. A. P. B (Esther) Oprins
    • 1
  • J. E (Hans) Korteling
    • 1
  1. 1.TNODelftThe Netherlands

Personalised recommendations