Skip to main content
SpringerLink
Log in

Assessing Learning from, with, and in Games Revisited: A Heuristic for Emerging Methods and Commercial Off-the-Shelf Games

  • Chapter
  • First Online:
Game-Based Assessment Revisited

Part of the book series: Advances in Game-Based Learning ((AGBL))

Abstract

Technology systems and tools have evolved; current technologies are connected, pervasive, and smart. Digital systems like games have become more dynamic, emergent, and complex. In response to these advancements, considerable effort has been applied to understand how people learn from, with, and within game-based environments. Scientists continue to expand the perspectives and views on humans and their interactions with systems like video games. Further, the methods and tools available to frame research, capture and extract data from these environments, and contextualize the inferences from the patterns discovered among the data have also evolved. This chapter coalesces experience from research with video games informed by three key perspectives: (1) games are complex systems; (2) human-computer interaction is a viable framework to describe learning with and in these systems; and (3) process-oriented data extracted from games can be informed from an analytics perspective. Specifically, six principles are presented that may help researchers engage in studies that involve the process of learning. These emerged from several years of research and include several key questions associated with special considerations, challenges, or pitfalls when deciding on a system, research questions, framework, etc. Subsequently, the chapter offers a discussion of existing/ongoing research involving Bully, The Deed, and League of Legends, including (a) detailed descriptions of the games, (b) discussions of each relevant class of affordances (i.e., player, researcher, and developer), and (c) practical research examples, including the purpose, method, and strategies to analyze data. Finally, the ways the heuristic and its principles guided each of the research examples are discussed as a means to contextualize broader implications for learning, assessment, research, and design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Alexander, P. A. (1992). Domain knowledge: Evolving themes and emerging concerns. Educational Psychologist, 27(1), 33–51.

    Article  Google Scholar 

  • Alexander, P. A. (2003). The development of expertise: The journey from acclimation to proficiency. Educational Researcher, 32(8), 10–14.

    Article  Google Scholar 

  • Alexander, P. A., & Dochy, F. J. R. C. (1995). Conceptions of knowledge and beliefs: A comparison across varying cultural and educational communities. American Educational Research Journal, 32(2), 413–442.

    Article  Google Scholar 

  • Baek, Y. (2017). Game-based learning: Theory, strategies and performance outcomes. Hauppauge, NY: Nova Science Publishers.

    Google Scholar 

  • Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.

    Google Scholar 

  • Barab, S. A., Gresalfi, M., & Ingram-Goble, A. (2010). Transformational play: Using games to position person, continent, and context. Educational Researcher, 39(7), 525–536. https://doi.org/10.3102/0013189X10386593

    Article  Google Scholar 

  • Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018). Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills. Psychological Bulletin, 144(1), 77–110.

    Article  Google Scholar 

  • Bernacki, M. L., & Walkington, C. (2018). The role of situational interest in personalized learning. Journal of Educational Psychology, 110(6), 864–881. https://doi.org/10.1037/edu0000250

    Article  Google Scholar 

  • Carroll, J. M., & Campbell, R. L. (1999). Artifacts as psychological theories: The case of human-computer interaction. Behaviour & Information Technology, 8(4), 247–256.

    Article  Google Scholar 

  • Clark, D., Tanner-Smith, E., & Killingsworth, S. (2014). Digital games, design and learning: A systematic review and meta-analysis (executive summary). Menlo Park, CA: SRI International.

    Google Scholar 

  • Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2012), 661–686.

    Article  Google Scholar 

  • Design Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.

    Article  Google Scholar 

  • DiCerbo, K. E., & Behrens, J. T. (2012). Implications of the digital ocean on current and future assessment. In R. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predictions for the future (pp. 273–306). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  • Ferdig, R. E. (2008). Handbook of research on effective electronic gaming in education (Vol. I–III). Hershey, PA: Information Science Reference.

    Google Scholar 

  • Gaver, W. H. (1991). Technology affordances. In Proceedings of CHI’91 (pp. 79–84). New York, NY: ACM.

    Google Scholar 

  • Ge, X., & Ifenthaler, D. (2017). Designing engaging educational games and assessing engagement in game-based learning. In R. Zheng & M. K. Gardner (Eds.), Handbook of research on serious games for educational applications (pp. 255–272). Hershey, PA: IGI Global.

    Google Scholar 

  • Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing: Toward an ecological psychology (pp. 67–82). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Girard, C., Ecalle, J., & Magnan, A. (2013). Serious games as new educational tools: How effective are they? a meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3), 207–219.

    Article  Google Scholar 

  • Grabthegames Studios. (2015). The deed. Grabthegames Studios.

    Google Scholar 

  • Greeno, J. G. (1994). Gibson’s affordances. The Psychological Review, 101(2), 336–342.

    Article  Google Scholar 

  • Hilpert, J. C., & Marchand, G. C. (2018). Complex systems research in educational psychology: Aligning theory and method. Educational Psychologist, 53(3), 185–202. https://doi.org/10.1080/00461520.2018.1469411

    Article  Google Scholar 

  • Ifenthaler, D., Eseryel, D., & Ge, X. (2012). Assessment in game-based learning: Foundations, innovations, and perspectives. New York, NY: Springer.

    Book  Google Scholar 

  • Jonassen, D. H., Campbell, J. P., & Davidson, M. E. (1994). Learning with media: Restructuring the debate. Educational Technology Research and Development, 42(2), 31–39.

    Article  Google Scholar 

  • Ke, F. (2009). A qualitative meta-analysis of computer games as learning tools. In R. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (Vol. I, pp. 1–32). Hershey, PA: Information Science Reference.

    Google Scholar 

  • King, F. B., Young, M. F., Drivere-Richmond, K., & Schrader, P. G. (2001). Defining distance learning and distance education. AACE Journal, 9(1), 1–14.

    Google Scholar 

  • Krach, S. K., & McCreery, M. P. (2015). Technology and positive behavioral interventions and support: Evaluation, selection, and implementation of computer-based socioemotional training. In S. Y. Tettegah & D. Espelage (Eds.), Emotions and technology, communication of feelings for, with and through digital media: Emotions, technology, and behavior. Waltham, MA: Elsevier.

    Google Scholar 

  • Laughlin, D., & Marchuk, N. D. (2005). A guide to educational computer games for NASA. Washington, DC: National Aeronautics and Space Administration’s Office of the Chief Education Officer, Technology and Products Office.

    Google Scholar 

  • Lejacq, Y. (2013, October 24). How fast is fast? Some pro gamers make 10 moves per second. ABC News. Retrieved from https://www.nbcnews.com/technolog/how-fast-fast-some-pro-gamers-make-10-moves-second-8C11422946

  • Loh, C. S., Sheng, Y., & Ifenthaler, D. (Eds.). (2015). Serious games analytics. Methodologies for performance measurement, assessment, and improvement. New York, NY: Springer.

    Google Scholar 

  • Mace, W. M. (1977). James J. Gibson’s strategy for perceiving: Ask not what’s inside your head, but what your head’s inside of. In R. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing (pp. 43–65). Hillsdale, NJ: LEA.

    Google Scholar 

  • Marchand, G. C., & Hilpert, J. C. (2017). Complex dynamic systems approaches to educational research. The Journal of Experimental Education, 85(2), 173–174. https://doi.org/10.1080/00220973.2016.1247555

    Article  Google Scholar 

  • Marchand, G. C., & Hilpert, J. C. (2018). Design considerations for education scholars interested in complex systems research. Complicity, 15(1), 1–14.

    Article  Google Scholar 

  • Mayer, R. E. (2015). On the need for research evidence to guide the design of computer games for learning. Educational Psychologist, 50(4), 349–353. https://doi.org/10.1080/00461520.2015.1133307

    Article  Google Scholar 

  • McCreery, M. P., Krach, S. K., & Nolen, A. L. (2014). The protagonist and their avatar: Learner characteristics in a culture of simulation. The International Journal of Gaming and Computer-Mediated Simulations, 6(2), 30–37.

    Article  Google Scholar 

  • McCreery, M. P., Krach, S. K., Schrader, P. G., & Boone, R. (2012). Defining the virtual self: Personality, behavior and the psychology of embodiment. Computers in Human Behavior, 28(3), 976–983.

    Article  Google Scholar 

  • McCreery, M. P., Schrader, P. G., Krach, S. K., & Boone, R. (2013). A sense of self: The role of presence in virtual environments. Computers in Human Behavior, 29(4), 1635–1640.

    Article  Google Scholar 

  • McCreery, M. P., Vallett, D., & Clark, C. (2015). Social interaction in a virtual environment: Examining sociospatial interactivity and social presence using behavioral analytics. Computers in Human Behavior, 51, 203–206.

    Article  Google Scholar 

  • McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY: The Penguin Press.

    Google Scholar 

  • Mirza-Babaei, P., Long, S., Foley, E., & McAllister, G. (2011). Understanding the contribution of biometrics to games user research. In Proceedings of DiGRA 2011 Conference: Think Design Play (Vol. 6, pp. 1–13). Retrieved from http://www.digra.org/digital-library/publications/understanding-the-contribution-of-biometrics-to-games-user-research/

  • Mirza-Babaei, P., Nacke, L. E., Gregory, J., Collins, N., & Fitzpatrick, G. (2013). How does it play better? Exploring user testing and biometric storyboards in games user research. In Conference on Human Factors in Computing Systems - Proceedings (pp. 1499–1508). https://doi.org/10.1145/2470654.2466200

  • O’Brien, D. A., Lawless, K. A., & Schrader, P. G. (2010). A taxonomy of educational games: Genres and applications. In Y. K. Baek (Ed.), Gaming for classroom-based learning: Digital role playing as a motivator of study (pp. 1–23). Hershey, PA: Information Science Reference.

    Google Scholar 

  • Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58.

    Article  Google Scholar 

  • Reiser, R. A. (2001). The history of instructional design and technology: Part 1: A history of instructional media. Educational Technology Research and Development, 49(1), 53–64. https://doi.org/10.1007/BF02504506.

    Article  Google Scholar 

  • Riot Games. (2009). The league of legends. Los Angeles, CA: Riot Games.

    Google Scholar 

  • Rockstar Games. (2006). Bully. New York, NY: Rockstar Games.

    Google Scholar 

  • Salomon, G., Perkins, D. N., & Globerson, T. (1991). Partners in cognition: Extending human intelligence with intelligent technologies. Educational Researcher, 20, 2–9.

    Article  Google Scholar 

  • Schenk, S., Lech, R. K., & Suchan, B. (2017). Games people play: How video games improve probabilistic learning. Behavioural Brain Research, 335, 208–214. https://doi.org/10.1016/j.bbr.2017.08.027

    Article  Google Scholar 

  • Schrader, P. G. (2008). Learning in technology: Reconceptualizing immersive environments. AACE Journal, 16(4), 457–475.

    Google Scholar 

  • Schrader, P. G., Deniz, H., & Keilty, J. (2016). Breaking SPORE: Building instructional value in science education using a commercial, off-the shelf game. Journal of Learning and Teaching in Digital Age, 1(1), 32–46.

    Google Scholar 

  • Schrader, P. G., & Lawless, K. A. (2007). Dribble files: Methodologies to evaluate learning and performance in complex environments. Performance Improvement, 46(1), 40–48.

    Article  Google Scholar 

  • Schrader, P. G., & McCreery, M. P. (2012). Are all games the same? Examining three frameworks for assessing learning from, with, and in games. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 11–28). New York, NY: Springer.

    Chapter  Google Scholar 

  • Schrader, P. G., McCreery, M. P., & Vallett, D. B. (2017). Performance in situ. In Y. Baek (Ed.), Game-based learning: Theory, strategies and performance outcomes (pp. 421–437). Hauppauge, NY: Nova Science Publishers.

    Google Scholar 

  • Shaffer, D. W., Squire, K. R., Halverson, R., & Gee, J. P. (2005). Video games and the future of learning. The Phi Delta Kappan, 87(2), 104–111. https://doi.org/10.1177/003172170508700205

    Article  Google Scholar 

  • Shalizi, C. R. (2006). Methods and techniques of complex systems science: An overview. In T. S. Deisboeck & J. Y. Kresh (Eds.), Complex systems science in biomedicine (pp. 33–114). New York, NY: Springer.

    Chapter  Google Scholar 

  • Shute, V. J., Ke, F., & Wang, L. (2017). Assessment and adaptation in games. In P. Wouters & H. van Oostendorp (Eds.), Instructional techniques to facilitate learning and motivation of serious games. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-39298-1_4

    Chapter  Google Scholar 

  • Squire, K. D. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19–29. https://doi.org/10.3102/0013189X035008019

    Article  Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  • Tettegah, S. Y., McCreery, M. P., & Blumberg, F. C. (2015). Toward a framework for learning and digital games research. Educational Psychologist, 50(4), 253–257.

    Article  Google Scholar 

  • Vallett, D. (2016). Soft failure in educational and learning games. In R. Lamb & D. McMahon (Eds.), Educational and learning games: New research. New York, NY: Nova Science Publishers.

    Google Scholar 

  • van Merrienboer, J. J. G., & Kirschner, P. A. (2018). Ten steps to complex learning: A systematic approach to four-component instructional design (3rd ed.). New York, NY: Routledge.

    Google Scholar 

  • van Merrienboer, J. J. G., & Sluijsmans, D. A. (2009). Toward a synthesis of cognitive load theory, four-component instructional design, and self-directed learning. Educational Psychology Review, 29, 5566.

    Google Scholar 

  • Ventura, M., & Shute, V. (2013). The validity of a game-based assessment of persistence. Computers in Human Behavior, 29(6), 2568–2572. https://doi.org/10.1016/j.chb.2013.06.033

    Article  Google Scholar 

  • Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229–243.

    Article  Google Scholar 

  • Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311

    Article  Google Scholar 

  • Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., … Yukhymenko, M. (2012). Our princess is in another castle: A review of trends in serious gaming for education. Review of Educational Research, 82(1), 61–89.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Nevada, Las Vegas, Las Vegas, NV, USA

    P. G. Schrader, Michael P. McCreery, Mark C. Carroll & Danielle L. Head

  2. Lebanon Valley College, Annville, PA, USA

    Jeffrey R. Laferriere

Authors
  1. P. G. Schrader
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael P. McCreery
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark C. Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danielle L. Head
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeffrey R. Laferriere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. G. Schrader .

Editor information

Editors and Affiliations

  1. Learning, Design and Technology, University of Mannheim, Mannheim, Germany

    Dirk Ifenthaler

  2. Teaching Systems Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

    Yoon Jeon Kim

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Schrader, P.G., McCreery, M.P., Carroll, M.C., Head, D.L., Laferriere, J.R. (2019). Assessing Learning from, with, and in Games Revisited: A Heuristic for Emerging Methods and Commercial Off-the-Shelf Games. In: Ifenthaler, D., Kim, Y.J. (eds) Game-Based Assessment Revisited. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-15569-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15569-8_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15568-1

  • Online ISBN: 978-3-030-15569-8

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation