Advertisement

The Dynamical Analysis of Log Data Within Educational Games

  • Erica L. SnowEmail author
  • Laura K. Allen
  • Danielle S. McNamara
Part of the Advances in Game-Based Learning book series (AGBL)

Abstract

Games and game-based environments frequently provide users multiple trajectories and paths. Thus, users often have to make decisions about how to interact and behave during the learning task. These decisions are often captured through the use of log data, which can provide a wealth of information concerning students’ choices, agency, and performance while engaged within a game-based system. However, to analyze these changing data sets, researchers need to use methodologies that focus on quantifying fine-grained patterns as they emerge across time. In this chapter, we will consider how dynamical analysis techniques offer researchers a unique means of visualizing and characterizing nuanced decision and behavior patterns that emerge from students’ log data within game-based environments. Specifically, we focus on how three distinct types of dynamical methodologies, Random Walks, Entropy analysis, and Hurst exponents, have been used within the game-based system iSTART-2 as a form of stealth assessment. These dynamical techniques provide researchers a means of unobtrusively assessing how students behave and learn within game-based environments.

Keywords

Dynamics Stealth assessments Data visualization Game-based environments 

Notes

Acknowledgments

This research was supported in part by the Institute for Educational Sciences (IES R305G020018-02; R305G040046, R305A080589) and National Science Foundation (NSF REC0241144; IIS-0735682). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the IES or NSF. We would like to thank all of the members of the SoletLab for their assistance with data collection.

References

  1. Allen, L. K., Snow, E. L., & McNamara, D. S. (2014). The long and winding road: Investigating the differential writing patterns of high and low skilled writers. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 304–307).Google Scholar
  2. Arneodo, M., Arvidson, A., Badełek, B., Ballintijn, M., Baum, G., Beaufays, J., et al. (1995). Measurement of the proton and the deuteron structure functions. Physics Letters B, 364, 107–115.CrossRefGoogle Scholar
  3. Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008). Why students engage in “gaming the system” behavior in interactive learning environments. Journal of Interactive Learning Research, 19, 185–224.Google Scholar
  4. Bandt, C., & Pompe, B. (2002). Permutation entropy: A natural complexity measure for time series. Physical Review Letters, 88, 174102.CrossRefGoogle Scholar
  5. Benhamou, S., & Bovet, P. (1989). How animals use their environment: A new look at kinesis. Animal Behavior, 38, 375–383.CrossRefGoogle Scholar
  6. Berg, H. C. (Ed.). (1993). Random walks in biology. Princeton, NJ: Princeton University Press.Google Scholar
  7. Berger, A. L., Pietra, V. J. D., & Pietra, S. A. D. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22(1), 39–71.Google Scholar
  8. Clausius, R. (1865). The mechanical theory of heat—With its applications to the steam engine and to physical properties of bodies. London: John van Voorst.Google Scholar
  9. Collins, J. J., & De Luca, C. J. (1994). Random walking during quiet standing. Physical Review Letters, 73, 764–767.CrossRefGoogle Scholar
  10. Costa, M., Goldberger, A. L., & Peng, C. K. (2002). Multiscale entropy analysis of complex physiologic time series. Physical Review Letters, 89(6), 68–102.CrossRefGoogle Scholar
  11. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.Google Scholar
  12. Granic, I., & Hollenstein, T. (2003). Dynamic systems methods for models of developmental psychopathology. Development and Psychopathology, 15(03), 641–669.CrossRefGoogle Scholar
  13. Grossman, E. R. F. W. (1953). Entropy and choice time: The effect of frequency unbalance on choice-response. Quarterly Journal of Experimental Psychology, 5(2), 41–51.CrossRefGoogle Scholar
  14. Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2, 107–124.CrossRefGoogle Scholar
  15. Hurst, H. E. (1951). Long-term storage capacity of reservoirs. Transactions of the American Society of Civil Engineers, 116, 770–808.Google Scholar
  16. Jackson, G. T., & McNamara, D. S. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105, 1036–1049.CrossRefGoogle Scholar
  17. Johnson, W. L., Marsella, S., Mote, N., Viljhalmsson, H., Narayanan, S., & Choi, S. (2004). Tactical Language Training System: Supporting the rapid acquisition of foreign language and cultural skills. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC).Google Scholar
  18. Landauer, T. K., McNamara, D. S., Dennis, S. E., & Kintsch, W. E. (2007). Handbook of latent semantic analysis. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  19. Lobry, J. R. (1996). Asymmetric substitution patterns in the two DNA strands of bacteria. Molecular Biological Evolution, 13, 660–665.CrossRefGoogle Scholar
  20. Mandelbrot, B. B. (1982). The fractal geometry of nature. New York: Freeman.Google Scholar
  21. McNamara, D. S., Boonthum, C., Levinstein, I. B., & Millis, K. (2007). Evaluating self-explanations in iSTART: Comparing word-based and LSA algorithms. In T. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of Latent Semantic Analysis (pp. 227–241). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  22. McNamara, D. S., Jackson, G. T., & Graesser, A. C. (2010). Intelligent tutoring and games (ITaG). In Y. K. Baek (Ed.), Gaming for classroom-based learning: Digital role-playing as a motivator of study (pp. 44–65). Hershey, PA: IGI Global.CrossRefGoogle Scholar
  23. McNamara, D. S., Levinstein, I. B., & Boonthum, C. (2004). iSTART: Interactive strategy trainer for active reading and thinking. Behavioral Research Methods, Instruments, & Computers, 36, 222–233.CrossRefGoogle Scholar
  24. McNamara, D. S., O’Reilly, T., Rowe, M., Boonthum, C., & Levinstein, I. B. (2007). iSTART: A web-based tutor that teaches self-explanation and metacognitive reading strategies. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 397–420). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  25. Nelson, C. R., & Plosser, C. R. (1982). Trends and random walks in macroeconomic time series: Some evidence and implications. Journal of Monetary Economics, 10, 139–162.CrossRefGoogle Scholar
  26. O’Reilly, T. P., Sinclair, G. P., & McNamara, D. S. (2004). Reading strategy training: Automated versus live. In K. Forbus, D. Gentner, & T. Regier (Eds.), Proceedings of the 26th Annual Cognitive Science Society (pp. 1059–1064). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  27. Peng, C. K., Havlin, S., Hausdorff, J. M., Mietus, B. S. J., Stanley, H. E., & Goldberger, A. L. (1995). Fractal mechanisms and heart rate dynamics: Long-range correlations and their breakdown with disease. Journal of Electrophysiology, 28, 59–65.Google Scholar
  28. Rai, D., & Beck, J. (2012). Math learning environment with game-like elements: An experimental framework. International Journal of Game Based Learning, 2, 90–110.CrossRefGoogle Scholar
  29. Sabourin, J., Shores, L. R., Mott, B. W., & Lester, J. C. (2012). Predicting student self-regulation strategies in game-based learning environments. In Intelligent tutoring systems (pp. 141–150). Berlin, Germany: Springer.Google Scholar
  30. Shannon, C. E. (1951). Prediction and entropy of printed English. Bell System Technical Journal, 30, 50–64.CrossRefGoogle Scholar
  31. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. Computer Games and Instruction, 55, 503–524.Google Scholar
  32. Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning. In U. Ritterfeld, M. J. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295–321). Philadelphia, PA: Routledge/LEA.Google Scholar
  33. Snow, E. L., Allen, L. K., Jackson, G. T., & McNamara, D. S. (2014). Tracking choices: Computational analysis of learning trajectories. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 316–319).Google Scholar
  34. Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). Does agency matter?: Exploring the impact of controlled behaviors within a game-based environment. Computers & Education, 26, 378–392.CrossRefGoogle Scholar
  35. Snow, E. L., Allen, L. K., Russell, D. G., & McNamara, D. S. (2014). Who’s in control?: Categorizing nuanced patterns of behaviors within a game-based intelligent tutoring system. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 185–192).Google Scholar
  36. Snow, E. L., Jacovina, M. E., Allen, L. K., Dai. J., & McNamara, D. S. (2014). Entropy: A stealth assessment of agency in learning environments. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 241–244).Google Scholar
  37. Snow, E. L., Likens, A., Jackson, G. T., & McNamara, D. S. (2013). Students’ walk through tutoring: Using a random walk analysis to profile students. In S. K. D’Mello, R. A. Calvo, & A. Olney (Eds.), Proceedings of the 6th International Conference on Educational Data Mining (pp. 276–279). Berlin, Germany: Springer.Google Scholar
  38. Spires, H. A., Rowe, J. P., Mott, B. W., & Lester, J. C. (2011). Problem solving and game-based learning: Effects of middle grade students’ hypothesis testing strategies on learning outcomes. Journal of Educational Computing Research, 44(4), 453–472.CrossRefGoogle Scholar
  39. Taylor, R. S., O’Reilly, T., Rowe, M., & McNamara, D. S. (2006). Improving understanding of science texts: iSTART strategy training vs. web design control task. In R. Sun & N. Miyake (Eds.), Proceedings of the 28th Annual Conference of the Cognitive Science Society (pp. 2234–2239). Mahwah, NJ: Erlbaum.Google Scholar
  40. Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Self-organization of cognitive performance. Journal of Experimental Psychology: General, 132(3), 331–350.CrossRefGoogle Scholar
  41. Zhou, M. (2013). Using traces to investigate self-regulatory activities: A study of self-regulation and achievement goal profiles in the context of web search for academic tasks. Journal of Cognitive Education and Psychology, 12, 287–305.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Erica L. Snow
    • 1
    Email author
  • Laura K. Allen
    • 1
  • Danielle S. McNamara
    • 1
  1. 1.Arizona State UniversityTempeUSA

Personalised recommendations