Serious Games Analytics pp 3-29

Part of the Advances in Game-Based Learning book series (AGBL) | Cite as

Serious Games Analytics: Theoretical Framework

  • Christian Sebastian Loh
  • Yanyan Sheng
  • Dirk Ifenthaler

Abstract

“Serious Games” is a unique industry that is concerned with the training/learning performance assessment of its clients. It is one of three digital technology industries (along with digital games, and online learning) that are rapidly advancing into the arena of analytics. The analytics from these industries all came from the tracing of user-generated data as they interacted with the systems, but differed from one another in the primary purposes for such analytics. For example, the purpose of game analytics is to support the growth of digital (entertainment) games, while that of learning analytics is to support the online learning industries. Although some game and learning analytics can indeed be used in serious games, they lack specific metrics and methods that outline the effectiveness of serious games—an important feature to stakeholders. Serious Games Analytics need to provide (actionable) insights that are of values to the stakeholders—specific strategies/policies to improve the serious games, and to (re)train or remediate play-learners for performance improvement. Since the performance metrics from one industry are unlikely to transfer well into another industry, those that are optimal for use in the Serious Games industry must be properly identified as Serious Games Analytics—to properly measure, assess, and improve performance with serious games.

Keywords

Serious games analytics Game analytics Learning analytics Definition Theoretical foundation Human performance assessment 

References

  1. Abt, C. C. (1987). Serious games. Lanham, MD: University Press of America (Reprint).Google Scholar
  2. Advances in game-based learning. Geneva, Switzerland: Springer. doi:10.1007/978-3-319-05834-4_5.
  3. Aldrich, C. (2005). Learning by doing: A comprehensive guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. San Francisco: Pfeiffer.Google Scholar
  4. Alvarez, J., Djaouti, D., Rampnoux, O., & Alvarez, V. (2011). Serious games market: Some key figures (from 1950’s to 2000’s). Retrieved October 22, 2014, from http://serious.gameclassification.com/files/articles/sgc_report_03-11.pdf
  5. Ambient Insight. (2013). 2013-2018 North America mobile edugame market, Monroe, WA. Retrieved December 12, 2014, from http://www.ambientinsight.com/Resources/Documents/Ambient-Insight-2013-2018-North-America-Mobile-Edugame-Market-Abstract.pdf
  6. BankersLab. (2013). A smart guide to serious gaming. Retrieved December 22, 2014, from http://bankerslab.com/blogposts/a-smart-guide-to-serious-gaming-part-1/
  7. Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction, 2013, 11. doi:10.1155/2013/136864.Google Scholar
  8. Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19(1–2), 205–220. doi:10.1007/s10758-014-9223-7.CrossRefGoogle Scholar
  9. Byun, J. H., & Loh, C. S. (2015). Audial engagement: Effects of game sound on learner engagement in digital game-based learning environments. Computers in Human Behavior, 46, 129–138. doi:10.1016/j.chb.2014.12.052.CrossRefGoogle Scholar
  10. Canossa, A., Seif El-Nasr, M., & Drachen, A. (2013). Benefits of game analytics: Stakeholders, contexts and domains. In M. Seif El-Nasr, A. Drachen, & A. Canossa (Eds.), Game analytics: Maximizing the value of player data (pp. 41–52). London: Springer. doi:10.1007/978-1-4471-4769-5.CrossRefGoogle Scholar
  11. Caspian Learning. (2010). Improving navy recruits’ performance: A serious games study. Sunderland, England. Retrieved December 12, 2014, from http://www.caspianlearning.co.uk/
  12. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459. doi:10.3102/00346543053004445.CrossRefGoogle Scholar
  13. Clark, R. E. (1985a). Confounding in educational computing research. Journal of Educational Computing Research, 1(2), 137–148.CrossRefGoogle Scholar
  14. Clark, R. E. (1985b). Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses. Educational Technology Research and Development, 33(4), 249–262.Google Scholar
  15. Clark, R. E. (1992). Dangers in the evaluation of instructional media. Academic Medicine, 67(12), 819–820.CrossRefGoogle Scholar
  16. Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. doi:10.1007/BF02299088.CrossRefGoogle Scholar
  17. Crookall, D. (2010). Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898–920. doi:10.1177/1046878110390784.CrossRefGoogle Scholar
  18. Cukier, K. N., & Mayer-Schoenberger, V. (2013). The rise of big data: How it’s changing the way we think about the World. Foreign Affairs, 92(3), 28–40. Retrieved December 12, 2014, from http://www.foreignaffairs.com/articles/139104/kenneth-neil-cukier-and-viktor-mayer-schoenberger/the-rise-of-big-data.Google Scholar
  19. DeRosier, M. E., Craig, A. B., & Sanchez, R. P. (2012). Zoo U: A stealth approach to social skills assessment in schools. Advances in Human-Computer Interaction, 2012. doi:10.1155/2012/654791.
  20. Djaouti, D., Alvarez, J., Jessel, J.-P., & Rampnoux, O. (2011). Origins of serious games. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious games and edutainment applications (pp. 25–43). London: Springer. doi:10.1007/978-1-4471-2161-9_3.CrossRefGoogle Scholar
  21. Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-Herráiz, J.-J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380–392. doi:10.1016/j.compedu.2012.12.020.CrossRefGoogle Scholar
  22. Donaldson, S. I., & Grant-Vallone, E. J. (2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245–60. doi:10.1023/A:1019637632584.
  23. Drager, N. (2014). Gamification, game-based learning, and serious games: What’s the difference? Retrieved June 09, 2014, from http://www.roninsc.com/blog/2014/05/08/gamification-game-based-learning-and-serious-games-whats-the-difference/
  24. Foundation of American Scientists. (2006). Summit on educational games: Harnessing the power of video games for learning. Washington, DC.Google Scholar
  25. Gee, J. P. (2003). What video games have to teach us about learning and literacy (1st ed.). New York: Palgrave Macmillan.Google Scholar
  26. Gerber, H. R. (2012). Can education be gamified?: Examining gamification, education, and the future. Retrieved June 4, 2014, from http://www.academia.edu/2235680/Can_Education_be_Gamified_Examining_Gamification_Education_and_the_Future
  27. Grimshaw, M., Lindley, C. A., & Nacke, L. (2008). Sound and immersion in the first-person shooter: Mixed measurement of the player’s sonic experience. In Proceedings of the Audio Mostly Conference. Retrieved December 12, 2014, from http://wlv.openrepository.com/wlv/bitstream/2436/35995/2/Grimshaw_CGAMES07.pdf
  28. Hastings, N. B., & Tracey, M. W. (2004). Does media affect learning: Where are we now? TechTrends, 49(2), 28–30. doi:10.1007/BF02773968.
  29. Herold, B. (2013). Researchers see video games as testing, learning tools. Education Week, 32(37), 14–15.Google Scholar
  30. Hughes, A. (2014). In serious games, analytics are everything! Learning Solution Magazine. Retrieved December 12, 2014, from http://www.learningsolutionsmag.com/articles/1509/in-serious-games-analytics-are-everything
  31. Ifenthaler, D. (2015). Learning analytics. In J. M. Spector (Ed.), The SAGE encyclopedia of educational technology. Thousand Oaks, CA: Sage.Google Scholar
  32. Ifenthaler, D., Eseryel, D., & Ge, X. (Eds.). (2012). Assessment in game-based learning: Foundations, innovations, and perspectives. New York: Springer. doi:10.1007/978-1-4614-3546-4.Google Scholar
  33. Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning, 19(1–2), 221–240. doi:10.1007/s10758-014-9226-4.CrossRefGoogle Scholar
  34. IJsselsteijn, W., de Kort, Y., Poels, K., Jurgelionis, A., & Bellotti, F. (2007). Characterising and measuring user experiences in digital games. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (pp. 27–30).Google Scholar
  35. Kay, D., & van Harmelen, M. (2012). Analytics for the whole institution: Balancing strategy and tactics. JISC CETIS, 1(2), 1–31.Google Scholar
  36. Kirriemuir, J., & McFarlane, A. (2003). Use of computer and video games in the classroom. In Proceedings of the Level up Digital Games Research Conference. Utrecht, The Netherlands: Universiteit Utrecht.Google Scholar
  37. Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61, 179–221. doi:10.3102/00346543061002179.CrossRefGoogle Scholar
  38. Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19.CrossRefGoogle Scholar
  39. Krulak, Charles. (1997). Military thinking and decision making exercises (No. 1500.55). Washington, DC. Retrieved December 12, 2014, from http://www.marines.mil/Portals/59/Publications/MCO 1500.55.pdf
  40. Laamarti, F., Eid, M., & El Saddik, A. (2014). An overview of serious games. International Journal of Computer Games Technology, 2014, 15. doi:10.1155/2014/358152.CrossRefGoogle Scholar
  41. LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., & Kruschwitz, N. (2011). Big data, analytics and the path from insights to value. MIT Sloan Management Review, 52(2), 21–31.Google Scholar
  42. Loh, C. S. (2006). Designing online games assessment as “information trails”. In D. Gibson, C. Aldrich, & M. Prensky (Eds.), Games and simulation in online learning: Research and development frameworks (pp. 323–348). Hershey, PA: Idea Group. doi:10.4018/978-1-59904-304-3.ch016.Google Scholar
  43. Loh, C. S. (2012a). Improving the impact and return of investment of game-based learning. International Journal of Virtual and Personal Learning Environments, 4(1), 1–15. doi:10.4018/jvple.2013010101.CrossRefGoogle Scholar
  44. Loh, C. S. (2012b). Information trails: In-process assessment of game-based learning. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 123–144). New York: Springer. doi:10.1007/978-1-4614-3546-4.CrossRefGoogle Scholar
  45. Loh, C. S., Anantachai, A., Byun, J. H., & Lenox, J. (2007). Assessing what players learned in serious games: In situ data collection, information trails, and quantitative analysis. In Q. Mehdi (Ed.), Proceedings of the Computer Games: AI, Animation, Mobile, Educational & Serious Games Conference (CGAMES 2007), Louisville, KY (pp. 10–19). Wolverhampton, England: University of Wolverhampton.Google Scholar
  46. Loh, C. S., & Sheng, Y. (2015). Measuring expert-performance for serious games analytics: From data to insights. In C. S. Loh, Y. Sheng, & D. Ifenthaler (Eds.), Serious games analytics: Methodologies for performance measurement, assessment, and improvement. Advances in Game-Based Learning. Springer International Publishing Switzerland. doi: 10.1007/978-3-319-05834-4_5
  47. Long, P. D., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 31–40.Google Scholar
  48. McLeroy, C. (2008a, September). History of military gaming. Soldiers Magazine, 4–6. Retrieved December 12, 2014, from http://www.army.mil/article/11936/History_of_Military_gaming/
  49. McLeroy, C. (2008b, September). Improving “America’s army”. Soldiers Magazine, 7–9. Retrieved December 12, 2014, from http://www.army.mil/article/11935/
  50. Medler, B. (2011). Player dossiers: Analyzing gameplay data as a reward. Game Studies, 11(1). Retrieved December 12, 2014, from http://gamestudies.org/1101/articles/medler
  51. Michael, D., & Chen, S. (2005). Proof of learning: Assessment in serious games. Retrieved September 22, 2014, from http://www.gamasutra.com/view/feature/2433/proof_of_learning_assessment_in_.php
  52. Michael, D., & Chen, S. (2006). Serious games: Games that educate, train, and inform. Boston: Thomson Course Technology PTR.Google Scholar
  53. Miller, R. (2013). Facebook builds exabyte data centers for cold storage. Retrieved June 8, 2014, from http://www.datacenterknowledge.com/archives/2013/01/18/facebook-builds-new-data-centers-for-cold-storage/
  54. Moshirnia, A. (2007). The educational potential of modified video games. In Proceedings of InSITE 2007: Informing Science and Information Technology Conference (pp. 511–521), Ljubljana, Slovenia.Google Scholar
  55. Nickols, F. W. (2005). Why a stakeholder approach to evaluating training. Advances in Developing Human Resources, 7(1), 121–134. doi:10.1177/1523422304272175.CrossRefGoogle Scholar
  56. Preimesberger, C. (2014). Apple unveils iOS 8 with improved Siri, security, health care features. Retrieved June 7, 2014, from http://www.eweek.com/mobile/slideshows/apple-unveils-ios-8-with-improved-siri-security-health-care-features.html
  57. Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.Google Scholar
  58. Rao, R. (2003). From unstructured data to actionable intelligence. IT Professional, 5(6), 29–35. doi:10.1109/MITP.2003.1254966.CrossRefGoogle Scholar
  59. Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. J. D. (Eds.). (2010). The handbook of educational data mining (1st ed.). Boca Raton, FL: CRC Press.Google Scholar
  60. Salomon, G., & Clark, R. E. (1974). Re-examining the methodology of research on media and technology in education. Review of Educational Research, 47, 99–120.CrossRefGoogle Scholar
  61. Sandford, R., & Williamson, B. (2005). Games and learning: A handbook. Bristol, England: FutureLab.Google Scholar
  62. Sawyer, B., & Rejeski, D. (2002). Serious games: Improving public policy through game-based learning and simulation. Washington, DC.Google Scholar
  63. Sawyer, B. (2009). Foreword: From virtual U to serious game to something bigger. In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. xi–xvi). New York: Routledge.Google Scholar
  64. Schaffhauser, D. (2014). Students blow through wind power in free virtual lab. Retrieved June 4, 2014, from http://thejournal.com/articles/2014/05/29/students-blow-through-wind-power-in-free-virtual-lab.aspx
  65. Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Shum, S. B., & Ferguson, R., et al. (2011). Open learning analytics: An integrated & modularized platform. Retrieved December 12, 2014, from http://solaresearch.org/OpenLearningAnalytics.pdf
  66. Söderber, H. (1977). Den allvarsamma leken (“The serious game”). Stockholm, Sweden: LiberForlag (Reprint).Google Scholar
  67. Thawonmas, R., & Iizuka, K. (2008). Visualization of online-game players based on their action behaviors. International Journal of Computer Games Technology, 2008, 1–9. doi:10.1155/2008/906931.CrossRefGoogle Scholar
  68. Turse, N. (2003). Zap, zap, you’re dead…. Retrieved June 4, 2014, from http://www.tomdispatch.com/index.mhtml?pid=1012
  69. Van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless…. EDUCAUSE Review, 41(2), 16–30. Retrieved Sep 12, 2014 from http://www.educause.edu/ero/article/digital-game-based-learning-its-not-just-digital-natives-who-are-restless.Google Scholar
  70. Wallner, G., & Kriglstein, S. (2012). A spatiotemporal visualization approach for the analysis of gameplay data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12) (pp. 1115–1124). New York: ACM Press. doi:10.1145/2207676.2208558.
  71. Watkins, R., Leigh, D., Foshay, R., & Kaufman, R. (1998). Kirkpatrick plus: Evaluation and continuous improvement with a community focus. Educational Technology Research and Development, 46(4), 90–96. doi:10.1007/BF02299676.CrossRefGoogle Scholar
  72. Williams, D., Yee, N., & Caplan, S. (2008). Who plays, how much, and why? A behavioral player census of a virtual World. Journal of Computer Mediated Communication, 13(4), 993–1018.CrossRefGoogle Scholar
  73. Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. New York: O’Reilly Media.Google Scholar
  74. Zoeller, G. (2013). Game development telemetry in production. In M. Seif-El-Nasr, A. Drachen, & A. Canossa (Eds.), Game analytics: Maximizing the value of player data (pp. 111–136). London: Springer.CrossRefGoogle Scholar
  75. Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38(9), 25–32. doi:10.1109/MC.2005.297.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Christian Sebastian Loh
    • 1
  • Yanyan Sheng
    • 2
  • Dirk Ifenthaler
    • 3
    • 4
    • 5
  1. 1.Virtual Environment Lab (V-Lab)Southern Illinois UniversityCarbondaleUSA
  2. 2.Department of Counseling, Quantitative Methods, and Special EducationSouthern Illinois UniversityCarbondaleUSA
  3. 3.University of MannheimMannheimGermany
  4. 4.Curtin UniversityPerthAustralia
  5. 5.Deakin UniversityMelbourneAustralia

Personalised recommendations