Abstract
Technology systems and tools have evolved; current technologies are connected, pervasive, and smart. Digital systems like games have become more dynamic, emergent, and complex. In response to these advancements, considerable effort has been applied to understand how people learn from, with, and within game-based environments. Scientists continue to expand the perspectives and views on humans and their interactions with systems like video games. Further, the methods and tools available to frame research, capture and extract data from these environments, and contextualize the inferences from the patterns discovered among the data have also evolved. This chapter coalesces experience from research with video games informed by three key perspectives: (1) games are complex systems; (2) human-computer interaction is a viable framework to describe learning with and in these systems; and (3) process-oriented data extracted from games can be informed from an analytics perspective. Specifically, six principles are presented that may help researchers engage in studies that involve the process of learning. These emerged from several years of research and include several key questions associated with special considerations, challenges, or pitfalls when deciding on a system, research questions, framework, etc. Subsequently, the chapter offers a discussion of existing/ongoing research involving Bully, The Deed, and League of Legends, including (a) detailed descriptions of the games, (b) discussions of each relevant class of affordances (i.e., player, researcher, and developer), and (c) practical research examples, including the purpose, method, and strategies to analyze data. Finally, the ways the heuristic and its principles guided each of the research examples are discussed as a means to contextualize broader implications for learning, assessment, research, and design.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexander, P. A. (1992). Domain knowledge: Evolving themes and emerging concerns. Educational Psychologist, 27(1), 33–51.
Alexander, P. A. (2003). The development of expertise: The journey from acclimation to proficiency. Educational Researcher, 32(8), 10–14.
Alexander, P. A., & Dochy, F. J. R. C. (1995). Conceptions of knowledge and beliefs: A comparison across varying cultural and educational communities. American Educational Research Journal, 32(2), 413–442.
Baek, Y. (2017). Game-based learning: Theory, strategies and performance outcomes. Hauppauge, NY: Nova Science Publishers.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
Barab, S. A., Gresalfi, M., & Ingram-Goble, A. (2010). Transformational play: Using games to position person, continent, and context. Educational Researcher, 39(7), 525–536. https://doi.org/10.3102/0013189X10386593
Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018). Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills. Psychological Bulletin, 144(1), 77–110.
Bernacki, M. L., & Walkington, C. (2018). The role of situational interest in personalized learning. Journal of Educational Psychology, 110(6), 864–881. https://doi.org/10.1037/edu0000250
Carroll, J. M., & Campbell, R. L. (1999). Artifacts as psychological theories: The case of human-computer interaction. Behaviour & Information Technology, 8(4), 247–256.
Clark, D., Tanner-Smith, E., & Killingsworth, S. (2014). Digital games, design and learning: A systematic review and meta-analysis (executive summary). Menlo Park, CA: SRI International.
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2012), 661–686.
Design Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.
DiCerbo, K. E., & Behrens, J. T. (2012). Implications of the digital ocean on current and future assessment. In R. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predictions for the future (pp. 273–306). Charlotte, NC: Information Age Publishing.
Ferdig, R. E. (2008). Handbook of research on effective electronic gaming in education (Vol. I–III). Hershey, PA: Information Science Reference.
Gaver, W. H. (1991). Technology affordances. In Proceedings of CHI’91 (pp. 79–84). New York, NY: ACM.
Ge, X., & Ifenthaler, D. (2017). Designing engaging educational games and assessing engagement in game-based learning. In R. Zheng & M. K. Gardner (Eds.), Handbook of research on serious games for educational applications (pp. 255–272). Hershey, PA: IGI Global.
Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing: Toward an ecological psychology (pp. 67–82). Hillsdale, NJ: Lawrence Erlbaum.
Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates.
Girard, C., Ecalle, J., & Magnan, A. (2013). Serious games as new educational tools: How effective are they? a meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3), 207–219.
Grabthegames Studios. (2015). The deed. Grabthegames Studios.
Greeno, J. G. (1994). Gibson’s affordances. The Psychological Review, 101(2), 336–342.
Hilpert, J. C., & Marchand, G. C. (2018). Complex systems research in educational psychology: Aligning theory and method. Educational Psychologist, 53(3), 185–202. https://doi.org/10.1080/00461520.2018.1469411
Ifenthaler, D., Eseryel, D., & Ge, X. (2012). Assessment in game-based learning: Foundations, innovations, and perspectives. New York, NY: Springer.
Jonassen, D. H., Campbell, J. P., & Davidson, M. E. (1994). Learning with media: Restructuring the debate. Educational Technology Research and Development, 42(2), 31–39.
Ke, F. (2009). A qualitative meta-analysis of computer games as learning tools. In R. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (Vol. I, pp. 1–32). Hershey, PA: Information Science Reference.
King, F. B., Young, M. F., Drivere-Richmond, K., & Schrader, P. G. (2001). Defining distance learning and distance education. AACE Journal, 9(1), 1–14.
Krach, S. K., & McCreery, M. P. (2015). Technology and positive behavioral interventions and support: Evaluation, selection, and implementation of computer-based socioemotional training. In S. Y. Tettegah & D. Espelage (Eds.), Emotions and technology, communication of feelings for, with and through digital media: Emotions, technology, and behavior. Waltham, MA: Elsevier.
Laughlin, D., & Marchuk, N. D. (2005). A guide to educational computer games for NASA. Washington, DC: National Aeronautics and Space Administration’s Office of the Chief Education Officer, Technology and Products Office.
Lejacq, Y. (2013, October 24). How fast is fast? Some pro gamers make 10 moves per second. ABC News. Retrieved from https://www.nbcnews.com/technolog/how-fast-fast-some-pro-gamers-make-10-moves-second-8C11422946
Loh, C. S., Sheng, Y., & Ifenthaler, D. (Eds.). (2015). Serious games analytics. Methodologies for performance measurement, assessment, and improvement. New York, NY: Springer.
Mace, W. M. (1977). James J. Gibson’s strategy for perceiving: Ask not what’s inside your head, but what your head’s inside of. In R. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing (pp. 43–65). Hillsdale, NJ: LEA.
Marchand, G. C., & Hilpert, J. C. (2017). Complex dynamic systems approaches to educational research. The Journal of Experimental Education, 85(2), 173–174. https://doi.org/10.1080/00220973.2016.1247555
Marchand, G. C., & Hilpert, J. C. (2018). Design considerations for education scholars interested in complex systems research. Complicity, 15(1), 1–14.
Mayer, R. E. (2015). On the need for research evidence to guide the design of computer games for learning. Educational Psychologist, 50(4), 349–353. https://doi.org/10.1080/00461520.2015.1133307
McCreery, M. P., Krach, S. K., & Nolen, A. L. (2014). The protagonist and their avatar: Learner characteristics in a culture of simulation. The International Journal of Gaming and Computer-Mediated Simulations, 6(2), 30–37.
McCreery, M. P., Krach, S. K., Schrader, P. G., & Boone, R. (2012). Defining the virtual self: Personality, behavior and the psychology of embodiment. Computers in Human Behavior, 28(3), 976–983.
McCreery, M. P., Schrader, P. G., Krach, S. K., & Boone, R. (2013). A sense of self: The role of presence in virtual environments. Computers in Human Behavior, 29(4), 1635–1640.
McCreery, M. P., Vallett, D., & Clark, C. (2015). Social interaction in a virtual environment: Examining sociospatial interactivity and social presence using behavioral analytics. Computers in Human Behavior, 51, 203–206.
McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY: The Penguin Press.
Mirza-Babaei, P., Long, S., Foley, E., & McAllister, G. (2011). Understanding the contribution of biometrics to games user research. In Proceedings of DiGRA 2011 Conference: Think Design Play (Vol. 6, pp. 1–13). Retrieved from http://www.digra.org/digital-library/publications/understanding-the-contribution-of-biometrics-to-games-user-research/
Mirza-Babaei, P., Nacke, L. E., Gregory, J., Collins, N., & Fitzpatrick, G. (2013). How does it play better? Exploring user testing and biometric storyboards in games user research. In Conference on Human Factors in Computing Systems - Proceedings (pp. 1499–1508). https://doi.org/10.1145/2470654.2466200
O’Brien, D. A., Lawless, K. A., & Schrader, P. G. (2010). A taxonomy of educational games: Genres and applications. In Y. K. Baek (Ed.), Gaming for classroom-based learning: Digital role playing as a motivator of study (pp. 1–23). Hershey, PA: Information Science Reference.
Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58.
Reiser, R. A. (2001). The history of instructional design and technology: Part 1: A history of instructional media. Educational Technology Research and Development, 49(1), 53–64. https://doi.org/10.1007/BF02504506.
Riot Games. (2009). The league of legends. Los Angeles, CA: Riot Games.
Rockstar Games. (2006). Bully. New York, NY: Rockstar Games.
Salomon, G., Perkins, D. N., & Globerson, T. (1991). Partners in cognition: Extending human intelligence with intelligent technologies. Educational Researcher, 20, 2–9.
Schenk, S., Lech, R. K., & Suchan, B. (2017). Games people play: How video games improve probabilistic learning. Behavioural Brain Research, 335, 208–214. https://doi.org/10.1016/j.bbr.2017.08.027
Schrader, P. G. (2008). Learning in technology: Reconceptualizing immersive environments. AACE Journal, 16(4), 457–475.
Schrader, P. G., Deniz, H., & Keilty, J. (2016). Breaking SPORE: Building instructional value in science education using a commercial, off-the shelf game. Journal of Learning and Teaching in Digital Age, 1(1), 32–46.
Schrader, P. G., & Lawless, K. A. (2007). Dribble files: Methodologies to evaluate learning and performance in complex environments. Performance Improvement, 46(1), 40–48.
Schrader, P. G., & McCreery, M. P. (2012). Are all games the same? Examining three frameworks for assessing learning from, with, and in games. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 11–28). New York, NY: Springer.
Schrader, P. G., McCreery, M. P., & Vallett, D. B. (2017). Performance in situ. In Y. Baek (Ed.), Game-based learning: Theory, strategies and performance outcomes (pp. 421–437). Hauppauge, NY: Nova Science Publishers.
Shaffer, D. W., Squire, K. R., Halverson, R., & Gee, J. P. (2005). Video games and the future of learning. The Phi Delta Kappan, 87(2), 104–111. https://doi.org/10.1177/003172170508700205
Shalizi, C. R. (2006). Methods and techniques of complex systems science: An overview. In T. S. Deisboeck & J. Y. Kresh (Eds.), Complex systems science in biomedicine (pp. 33–114). New York, NY: Springer.
Shute, V. J., Ke, F., & Wang, L. (2017). Assessment and adaptation in games. In P. Wouters & H. van Oostendorp (Eds.), Instructional techniques to facilitate learning and motivation of serious games. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-39298-1_4
Squire, K. D. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19–29. https://doi.org/10.3102/0013189X035008019
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.
Tettegah, S. Y., McCreery, M. P., & Blumberg, F. C. (2015). Toward a framework for learning and digital games research. Educational Psychologist, 50(4), 253–257.
Vallett, D. (2016). Soft failure in educational and learning games. In R. Lamb & D. McMahon (Eds.), Educational and learning games: New research. New York, NY: Nova Science Publishers.
van Merrienboer, J. J. G., & Kirschner, P. A. (2018). Ten steps to complex learning: A systematic approach to four-component instructional design (3rd ed.). New York, NY: Routledge.
van Merrienboer, J. J. G., & Sluijsmans, D. A. (2009). Toward a synthesis of cognitive load theory, four-component instructional design, and self-directed learning. Educational Psychology Review, 29, 5566.
Ventura, M., & Shute, V. (2013). The validity of a game-based assessment of persistence. Computers in Human Behavior, 29(6), 2568–2572. https://doi.org/10.1016/j.chb.2013.06.033
Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229–243.
Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311
Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., … Yukhymenko, M. (2012). Our princess is in another castle: A review of trends in serious gaming for education. Review of Educational Research, 82(1), 61–89.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Schrader, P.G., McCreery, M.P., Carroll, M.C., Head, D.L., Laferriere, J.R. (2019). Assessing Learning from, with, and in Games Revisited: A Heuristic for Emerging Methods and Commercial Off-the-Shelf Games. In: Ifenthaler, D., Kim, Y.J. (eds) Game-Based Assessment Revisited. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-15569-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-15569-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15568-1
Online ISBN: 978-3-030-15569-8
eBook Packages: EducationEducation (R0)