Using Epistemic Networks with Automated Codes to Understand Why Players Quit Levels in a Learning Game

  • Shamya KarumbaiahEmail author
  • Ryan S. Baker
  • Amanda Barany
  • Valerie Shute
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1112)


Understanding why students quit a level in a learning game could inform the design of appropriate and timely interventions to keep students motivated to persevere. In this paper, we study student quitting behavior in Physics Playground (PP) – a Physics game for secondary school students. We focus on student cognition that can be inferred from their interaction with the game. PP logs meaningful and crucial student behaviors relevant to physics learning in real time. The automatically generated events in the interaction log are used as codes for quantitative ethnography analysis. We study epistemic networks from five levels to study how the temporal interconnections between the events are different for students who quit the game and those who did not. Our analysis revealed that students who quit over-rely on nudge actions and tend to settle on a solution more quickly than students who successfully complete a level, often failing to identify the correct agent and supporting objects to solve the level.


Learning game Quitting behavior Epistemic network analysis Automated codes Interaction log 


  1. 1.
    Shute, V.J., Ventura, M., Kim, Y.J.: Assessment and learning of qualitative physics in newton’s playground. J. Educ. Res. 106(6), 423–430 (2013)CrossRefGoogle Scholar
  2. 2.
    Lomas, D., Patel, K., Forlizzi, J.L., Koedinger, K.R.: Optimizing challenge in an educational game using large-scale design experiments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 89–98. ACM, New York (2013)Google Scholar
  3. 3.
    Karumbaiah, S., Rahimi, S., Baker, R.S, Shute, V.J., D’Mello, S.: Is student frustration in learning games more associated with game mechanics or conceptual understanding? In: Kay, J., Luckin, R. (eds.) 13th International Conference of Learning Sciences, London, UK, vol. 3, pp. 1385–1386 (2018)Google Scholar
  4. 4.
    Baker, R.S.J., Mitrović, A., Mathews, M.: Detecting gaming the system in constraint-based tutors. In: De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 267–278. Springer, Heidelberg (2010). Scholar
  5. 5.
    Shaffer, D.W.: Quantitative Ethnography. Cathcart Press (2017)Google Scholar
  6. 6.
    Owen, V.E.: Capturing in-game learner trajectories with ADAGE (assessment data aggregator for game environments): a cross-method analysis. Doctoral dissertation, University of Wisconsin-Madison, Madison, WI (2014)Google Scholar
  7. 7.
    Karumbaiah, S., Baker, R.S., Shute, V.: Predicting quitting in students playing a learning game. In: Boyer, K.E., Yudelson, M. (eds.) Proceedings of the 11th International Conference on Educational Data Mining, pp. 167–176 (2018)Google Scholar
  8. 8.
    Shaffer, D.W., Collier, W., Ruis, A.R.: A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Anal. 3(3), 9–45 (2016)CrossRefGoogle Scholar
  9. 9.
    Arastoopour, G., Shaffer, D.W., Swiecki, Z., Ruis, A.R., Chesler, N.C.: Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. Int. J. Eng. Educ. 32(2) (2016)Google Scholar
  10. 10.
    Knight, S., Arastoopour, G., Williamson Shaffer, D., Buckingham Shum, S., Littleton, K.: Epistemic networks for epistemic commitments. In: Polman, J.L., et al. (eds.) Learning and Becoming in Practice: The International Conference of the Learning Sciences, Boulder, CO, vol. 1, pp. 150–157 (2014)Google Scholar
  11. 11.
    Beck, J.E., Gong, Y.: Wheel-spinning: students who fail to master a skill. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 431–440. Springer, Heidelberg (2013). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Shamya Karumbaiah
    • 1
    Email author
  • Ryan S. Baker
    • 1
  • Amanda Barany
    • 2
  • Valerie Shute
    • 3
  1. 1.University of PennsylvaniaPhiladelphiaUSA
  2. 2.Drexel UniversityPhiladelphiaUSA
  3. 3.Florida State UniversityTallahasseeUSA

Personalised recommendations