Assessing Game-Based Mathematics Learning in Action
Digital learning environments emphasize learning in action. Because knowledge is present in what learners do, how they do it, what tools they use, and how they communicate in and about their doing, it is important to assess knowledge production in context and learning in action. Via a design-based research approach, we explored the feasibility and validity of using the evidence-centered design approach and Bayesian networks to assess mathematical learning in action in a game-based learning environment. We iteratively tested the hypothesized assessment models and alternative approaches of exploiting game-based performance data, with longitudinal data sets collected during the course of 42 gaming sessions across 3 academic semesters. The investigation illustrated the design and implementation heuristics related to the game-based learning-in-action assessment. The emerged learning-in-action assessment evolves around four major operational practices: (a) domain competency modeling along with core game mechanics conceptualization; (b) developing task models and the Q-matrix; (c) developing the game log that encompasses performance data capturing, pattern recognition, and observables extraction; and (d) training, substantiating, and comparing statistical models for data processing and assessment implementation.
KeywordsGame-based learning Mathematics Real-time assessment Bayesian network Learning in action
The work reported in this chapter was supported by the National Science Foundation, grant no. 1318784. Any opinions, findings, and conclusions or recommendations expressed in these materials are those of the author and do not necessarily reflect the views of the National Science Foundation.
- Ayers, E., & Junker, B. W. (2006, July). Do skills combine additively to predict task difficulty in eighth grade mathematics. In Educational data mining: Papers from the AAAI workshop. Menlo Park, CA: AAAI.Google Scholar
- Carnoy, M., & Rothstein, R. (2013). What do international tests really show us about U.S. student performance? Washington, DC: Economic Policy Institute.Google Scholar
- Dede, C. (2012, May). Interweaving assessments into immersive authentic simulations: Design strategies for diagnostic and instructional insights. In Invitational Research Symposium on Technology Enhanced Assessments. Retrieved from http://www.ets.org/Media/Research/pdf/session4-dede-paper-tea2012.pdf
- Ke, F. (2019). Mathematical problem solving and learning in an architecture-themed epistemic game. Educational Technology Research and Development. https://doi.org/10.1007/s11423-018-09643-2
- Ke, F., & Clark, K. M. (2019). Game-based multimodal representations and mathematical problem solving. International Journal of Science and Mathematics Education. https://doi.org/10.1007/s10763-018-9938-3
- Klopfer, E., Osterweil, S., & Salen, K. (2009). Moving learning games forward. Cambridge, MA: The Education Arcade. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.687.5017&rep=rep1&type=pdfGoogle Scholar
- Levy, R. (2014). Dynamic Bayesian network modeling of game based diagnostic assessments (CRESST Report 837). National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved from https://files.eric.ed.gov/fulltext/ED555714.pdf
- Manjarres, A. V., Sandoval, L. G. M., & Suárez, M. S. (2018). Data mining techniques applied in educational environments: Literature review. Digital Education Review, 33, 235–266.Google Scholar
- Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4(1), 11–48.Google Scholar
- Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology & Society, 17(4), 49–64.Google Scholar
- Pardos, Z., Heffernan, N., Ruiz, C., & Beck, J. (2008, June). The composition effect: Conjunctive or compensatory? An analysis of multi-skill math questions in ITS. In R. de Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining 2008 proceedings (pp. 147–156).Google Scholar