Advertisement

Learning Analytics as an Assessment Tool in Serious Games: A Review of Literature

  • Min Liu
  • Jina Kang
  • Sa Liu
  • Wenting Zou
  • Jeff Hodson
Chapter

Abstract

The purpose of this chapter is to conduct a systematic review of research on studies using analytics (particularly in-game data such as logs) in serious games (SG) to understand what research has been conducted and what research evidences there are in using analytics in SG to support teaching and learning. The findings of this review showed learner performance and game design strategies were the two most common researched topics. Other topics included motivation and engagement, student behavior, problem solving, learner progress trajectories, and student collaboration. In addressing students’ learning performance, more studies reported that SG had a positive impact on learning; and many highlighted the importance of game design. Some of the studies reviewed also indicated the challenges for researchers to use in-game dynamic data as a research measure. Several trends are identified and implications for future research are discussed.

Keywords

Serious games Learning analytics In-game log data Literature review Analytics as an assessment tool 

References1

  1. *Burton, B. G., Martin, B. N.: Learning in 3d virtual environments: collaboration and knowledge spirals. J. Educ. Comput. Res. 43(2), 259–273 (2010)Google Scholar
  2. *Cagiltay, N. E., Ozcelik, E., Ozcelik, N. S.: The effect of competition on learning in games. Comput. Educ. 87, 35–41 (2015).Google Scholar
  3. Cappiello, C., Matera, M., Picozzi, M., Sprega, G., Barbagallo, D., & Francalanci, C.. DashMash: a mashup environment for end user development. In: Auer, S., Díaz, O. & Papadopoulos, G. A. (eds.). Web Engineering, vol. 6757, pp. 152–166. Springer, Berlin, Heidelberg.  http://dx.doi.org/10.1007/978–3- 642-22233-7 (2011)
  4. *Cheng, M. T., Lin, Y. W., & She, H. C.: Learning through playing virtual age: exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters. Comput. Educ. 86, 18–29 (2015).Google Scholar
  5. Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper Perennial, New York (1991)Google Scholar
  6. Duggan, M.: Gaming and gamers. Retrieved from: http://www.pewinternet.org/2015/12/15/gaming-and-gamers/ (2015, December 15)
  7. *Forsyth, C. M., Graesser, A. C., Jr, Pavlik, P., Cai, Z., Butler, H., Halpern, D., & Millis, K: Operation ARIES!: methods, mystery, and mixed models: discourse features predict affect in a serious game. J. Educ. Data Mining. 5(1), 147–189 (2013)Google Scholar
  8. *Gauthier, A., Corrin, M., & Jenkinson, J.: Exploring the influence of game design on learning and voluntary use in an online vascular anatomy study aid. Comput. Educ. 87, 24–34 (2015)Google Scholar
  9. *Hämäläinen, R.: Designing and evaluating collaboration in a virtual game environment for vocational learning. Comput. Educ. 50(1), 98–109 (2008)Google Scholar
  10. Hart, C.: Doing a Literature Review: Releasing the Social Science Research Imagination. Sage Publications, London (1999)Google Scholar
  11. *Hou, H. T.: Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Comput. Educ. 58(4), 1225–1233 (2012)Google Scholar
  12. *Kerr, D.: Using data mining results to improve educational video game design. J. Educ. Data Mining 7(3), 1–17 (2015)Google Scholar
  13. *Kerr, D., Chung, G. K.: Identifying key features of student performance in educational video games and simulations through cluster analysis. J. Educ. Data Mining 4(1), 144–182 (2012)Google Scholar
  14. *Kiili, K.: Content creation challenges and flow experience in educational games: the IT-Emperor case Internet High. Educ. 8(3), 183–198 (2005)Google Scholar
  15. *Klopfer, E., Sheldon, J., Perry, J., & Chen, V. H.: Ubiquitous games for learning (UbiqGames): weatherlings, a worked example. J. Comput. Assist. Learn. 28(5), 465–476 (2012)Google Scholar
  16. Kozma, R.B., Russell, J.: Multimedia and understanding: expert and novice responses to different representations of chemical phenomena. J. Res. Sci. Teach. 34(9), 949–968 (1997)CrossRefGoogle Scholar
  17. *Liao, C. C., Chen, Z. H., Cheng, H. N., Chen, F. C., & Chan, T. W.: My-Mini-Pet: a handheld pet-nurturing game to engage students in arithmetic practices. J. Comput. Assist. Learn. 27(1), 76–89 (2011)Google Scholar
  18. *Liu, C. C., Cheng, Y. B., & Huang, C. W.: The effect of simulation games on the learning of computational problem solving. Comput. Educ. 57(3), 1907–1918 (2011)Google Scholar
  19. Liu, M., Geurtz, R., Karam, A., Navarrete, C., Scordino, R.: Research on mobile learning in adult education. In: Kinuthia, W., Marshall, S. (eds.) On the Move: Mobile Learning for Development. Information Age Publishing, Charlotte (2013)Google Scholar
  20. Liu, M., Scordino, R., Geurtz, R., Navarrete, C., Ko, Y.J., Lim, M.H.: A look at research on mobile learning in k-12 education from 2007 to the present. J. Res. Technol. Educ. 46(4), 325–372 (2014)CrossRefGoogle Scholar
  21. Liu, M., Kang, J., Lee, J., Winzeler, E., Liu, S.: Examining through visualization what tools learners access as they play a serious game for middle school science. In: Loh, C.S., Sheng, Y., Ifenthaler, D. (eds.) Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement. Springer International Publishing, New York (2015)Google Scholar
  22. *Liu, M., Lee, J., Kang, J, & Liu, S: What we can learn from the data: a multiple-case study examining behavior patterns by students with different characteristics in using a serious game. Technol. Knowl. Learn. J., 21(1), 33–57 (2016). doi: 10.1007/s10758-015-9263-7
  23. *Loh, C. S., Sheng, Y.: Measuring the (dis-) similarity between expert and novice behaviors as serious games analytics. Educ. Inf. Technol. 20(1), 5–19 (2013)Google Scholar
  24. *Loh, C. S., Sheng, Y.: Maximum Similarity Index (MSI): a metric to differentiate the performance of novices vs. multiple-experts in serious games. Comput. Hum. Behav. 39, 322–330 (2014)Google Scholar
  25. *Loh, C. S., Sheng, Y., & Li, I. H.: Predicting expert–novice performance as serious games analytics with objective-oriented and navigational action sequences. Comput. Hum. Behav. 49, 147–155 (2015)Google Scholar
  26. *Minović, M., Milovanović, M., Šošević, U., & González, M.Á.C.: Visualisation of student learning model in serious games. Comput. Hum. Behav. 47, 98–107 (2015)Google Scholar
  27. Mishra, J., Zinni, M., Bavelier, D., Hillyard, S.A.: Neural basis of superior performance of action videogame players in an attention-demanding task. J. Neurosci. 31(3), 992–998 (2011)CrossRefGoogle Scholar
  28. Reese, D.D., Seward, R.J., Tabachnick, B.G., Hitt, B., Harrison, A., McFarland, L.: Timed Report measures learning: game-based embedded assessment. In: Ifenthaler, D., Eseryel, D., Ge, X. (eds.) Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives, pp. 145–172. Springer, New York (2012)CrossRefGoogle Scholar
  29. *Reese, D. D., Tabachnick, B. G., & Kosko, R. E.: Video game learning dynamics: actionable measures of multidimensional learning trajectories. Br. J. Educ. Technol. 46(1), 98–122 (2015)Google Scholar
  30. Rideout, V. J., Foehr, U. G., Roberts, D. F.: Generation M2: media in the lives of 8- to 18-year-olds. Kaiser Family Foundation. Retrieved from http://kff.org/other/poll-finding/report-generation-m2-media-in-the-lives/ (2010, January)
  31. *Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S.: Does agency matter?: exploring the impact of controlled behaviors within a game-based environment. Comput. Educ. 82, 378–392 (2015)Google Scholar
  32. *Spires, H. A., Rowe, J. P., Mott, B. W., & Lester, J. C.: Problem solving and game-based learning: effects of middle grade students’ hypothesis testing strategies on learning outcomes. J. Educ. Comput. Res. 44(4), 453–472 (2011)Google Scholar
  33. Strauss, A.C., Corbin, J.M.: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage Publications, Newbury Park (1990)Google Scholar
  34. *Sun, C. T., Wang, D. Y., & Chan, H.L.: How digital scaffolds in games direct problem-solving behaviors. Comput. Educ. 57(3), 2118–2125 (2011)Google Scholar
  35. U.S. Department of Education, Office of Educational Technology: Enhancing teaching and learning through educational data mining and learning analytics: an issue brief. Retrieved from https://tech.ed.gov/wp-content/uploads/2014/03/edm-la-brief.pdf (2012)
  36. van Barneveld, A., Arnold, K.E., & Campbell, J.P.: Analytics in higher education: establishing a common language. EDUCAUSE Learning Initiative. Retrieved from https://qa.itap.purdue.edu/learning/docs/research/ELI3026.pdf (2012)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Min Liu
    • 1
  • Jina Kang
    • 1
  • Sa Liu
    • 1
  • Wenting Zou
    • 1
  • Jeff Hodson
    • 1
  1. 1.University of Texas at AustinAustinUSA

Personalised recommendations