Skip to main content

Approaches to illuminate content-specific gameplay decisions using open-ended game data

Educational Technology Research and Development volume 69, pages 1135–1154 (2021)Cite this article

Abstract

Games can be rich environments for learning and can elicit evidence of students’ conceptual understanding and inquiry processes. Illuminating students’ content-specific gameplay decisions, or methods of completing game tasks related to a certain domain, requires a context that is open-ended enough for students to make choices that demonstrate their thinking. Doing this also requires rich log data and methods of Game Learning Analytics (GLA) that are granular enough to look at the specific choices most relevant to that context and domain. This paper presents research done on student exploration of high school level Mendelian genetics in a multiplayer online game called The Radix Endeavor. The study uses three approaches to identify content-specific gameplay decisions and distinguish players utilizing different methods, looking at actions and tool use, play patterns and player types, and tool input patterns. In the context of the selected game quest, the three approaches were found to yield insights into different ways that students complete tasks in genetics, suggesting the potential for a set of more generalized guiding questions in the GLA field that could be adopted by learning games designers and data scientists to convey information about content-specific gameplay decisions in learning games.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  • Alonso-Fernández, C., Calvo, A., Freire, M., Martinez-Ortiz, I., & Fernandez-Manjon, B. (2017). Systematizing game learning analytics for serious games. Paper presented at the—2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 1111–1118). https://doi.org/10.1109/EDUCON.2017.7942988

  • Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2019). Applications of data science to game learning analytics data: A systematic literature review. Computers and Education, 141, 103612. https://doi.org/10.1016/j.compedu.2019.103612.

    Article  Google Scholar 

  • Awang-Kanak, F., Masnoddin, M., Matawali, A., Daud, M. A., & Jumat, N. R. (2016). Difficulties experience by science foundation students on basic mendelian genetics topic: A preliminary study. Transactions on Science and Technology, 3(1–2), 283–290.

    Google Scholar 

  • Aznar, M. M., & Orcajo, T. I. (2005). Solving problems in genetics. International Journal of Science Education, 27(1), 101–121. https://doi.org/10.1080/09500690410001673801.

    Article  Google Scholar 

  • Bado, N. (2019). Game-based learning pedagogy: A review of the literature. Interactive Learning Environments. https://doi.org/10.1080/10494820.2019.1683587.

    Article  Google Scholar 

  • Bahar, M., Johnstone, A. H., & Hansell, M. H. (1999). Revisiting learning difficulties in biology. Journal of Biological Education, 33(2), 84–86. https://doi.org/10.1080/00219266.1999.9655648.

    Article  Google Scholar 

  • Butler, E., & Banerjee, R. (2014). Visualizing progressions for education and game design (white paper). Retrieved from https://www.ericbutler.net/assets/papers/unpub2014_visualization.pdf.

  • Chaudy, Y., Connolly, T., & Hainey, T. (2014). Learning analytics in serious games: A review of the literature. Paper presented at the ECAET 2014: European Conference in the Applications of Enabling Technologies. United Kingdom

  • Cheng, M.-T. (2020). The radix endeavor dataset. figshare. https://doi.org/10.6084/m9.figshare.12442469.v1.

  • Chu, Y., & Reid, N. (2012). Genetics at school level: Addressing the difficulties. Research in Science and Technological Education, 30(3), 285–309. https://doi.org/10.1080/02635143.2012.732059.

    Article  Google Scholar 

  • Collins, A., & Stewart, J. H. (1989). The knowledge structure of mendelian genetics. The American Biology Teacher, 51(3), 143–149.

    Article  Google Scholar 

  • Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., & players, F. (2010). Predicting protein structures with a multiplayer online game. Nature, 466(7307), 756–760. https://doi.org/10.1038/nature09304.

    Article  Google Scholar 

  • Duncan, R. G., & Reiser, B. J. (2007). Reasoning across ontologically distinct levels: Students’ understandings of molecular genetics. Journal of Research in Science Teaching, 44(7), 938–959. https://doi.org/10.1002/tea.20186.

    Article  Google Scholar 

  • Etobro, A. B., & Banjoko, S. O. (2017). Misconceptions of genetics concepts among pre-service teachers. Global Journal of Educational Research, 16(2), 121–128.

    Article  Google Scholar 

  • Freire, M., Serrano-Laguna, Á, Manero, B., Martínez-Ortiz, I., Moreno-Ger, P., & Fernández-Manjón, B. (2016). Game learning analytics: Learning analytics for serious games. Learning, design, and technology (pp. 1–29). AG, Switzerland: Springer Nature.

  • Institute of play. (2013). Playtime online 21: Digging into data with SimCityEDU. Retrieved from https://www.youtube.com/watch?v=Y5_7Y7wKE6A&feature=share.

  • Johnstone, A. H. (1991). Why is science difficult to learn? things are seldom what they seem. Journal of Computer Assisted Learning, 7(2), 75–83.

    Article  Google Scholar 

  • Juul, J. (2011). Half-real: Video games between real rules and fictional worlds. . Cambridge: MIT Press.

    Google Scholar 

  • Karagoz, M., & Cakir, M. (2011). Problem solving in genetics: Conceptual and procedural difficulties. Educational Sciences Theory and Practice, 11(3), 1668–1674.

    Google Scholar 

  • Ke, F. (2016). Designing and integrating purposeful learning in game play: A systematic review. Educational Technology Research and Development, 64(2), 219–244. https://doi.org/10.1007/s11423-015-9418-1.

    Article  Google Scholar 

  • Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in river city, a multi-user virtual environment. Journal of Science Education and Technology, 16(1), 99–111. https://doi.org/10.1007/s10956-006-9038-y.

    Article  Google Scholar 

  • Klopfer, E., Haas, J., Osterweil, S., & Rosenheck, L. (2018). Resonant games. Cambridge: MIT Press.

  • Knippels, M. P. J., & Waarlo, A. J. (2018). Development, uptake, and wider applicability of the yo-yo strategy in biology education research: A reappraisal. Education Sciences, 8(3), 129.

    Article  Google Scholar 

  • Knippels, M. P. J., Waarlo, A. J., & Boersma, K. T. (2005). Design criteria for learning and teaching genetics. Journal of Biological Education, 39(3), 108–112. https://doi.org/10.1080/00219266.2005.9655976.

    Article  Google Scholar 

  • Lewis, J., & Kattmann, U. (2004). Traits, genes, particles and information: Re-visiting students’ understandings of genetics. International Journal of Science Education, 26(2), 195–206. https://doi.org/10.1080/0950069032000072782.

    Article  Google Scholar 

  • LiuKangLiuZouHodson, M. J. S. W. J. (2017). Learning analytics as an assessment tool in serious games: A review of literature. In M. A. MaOikonomou (Ed.), Serious games and edutainment applications. (pp. 537–563). Springer.

    Google Scholar 

  • Loh, C. S., Sheng, Y., & Ifenthaler, D. (2015a). Serious games analytics: Theoretical framework. In C. S. Loh, Y. Sheng, & D. Ifenthaler (Eds.), Serious game analytics: Methodologies for performance measurement, assessment, and improvement Springer.

    Chapter  Google Scholar 

  • Loh, C. S., Sheng, Y., & Ifenthaler, D. (2015b). Serious games analytics: Methodologies for performance measurement, assessment, and improvement. . Springer.

    Book  Google Scholar 

  • NHGRI. (n.d.). Phenotype. Retrieved from https://www.genome.gov/genetics-glossary/Phenotype

  • Orcajo, T. I., & Aznar, M. M. (2005). Solving problems in genetics II: Conceptual restructuring. International Journal of Science Education. 27(12), 1495–1519. https://doi.org/10.1080/09500690500186584.

    Article  Google Scholar 

  • O'Rourke, E., Butler, E., Liu, Y., Ballweber, C., & Popovic, Z. (2013). The effects of age on player behavior in educational games. Paper presented at the The 8th International Conference on the Foundations of Digital Games (pp. 158–165)

  • Owen, V. E., & Baker, R. S. (2019). Learning analytics for games. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning. (pp. 513–536). MIT.

    Google Scholar 

  • Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58. https://doi.org/10.1016/j.chb.2016.05.023.

    Article  Google Scholar 

  • Rosenheck, L., Gordon-Messer, S., Clarke-Midura, J., & Klopfer, E. (2016). Design and implementation of an MMO: Approaches to support inquiry learning with games. In D. Russell, & J. M. Laffey (Eds.), Handbook of research on gaming trends in P-12 education (pp. 33-54). Hershey, PA: IGI Global

  • Rowe, E., Asbell-Clarke, J., & Baker, R. S. (2015). Serious games analytics to measure implicit science learning. In C. S. Loh, Y. Sheng & D. Ifenthaler (Eds.), Serious games analytics: Methodologies for performance measurement, assessment, and improvement (pp. 343-360). Cham: Springer International Publishing. doi:https://doi.org/10.1007/978-3-319-05834-4_15 Retrieved from https://doi.org/https://doi.org/10.1007/978-3-319-05834-4_15

  • Shaw, K. R. M., Van Horne, K., Zhang, H., & Boughman, J. (2008). Essay contest reveals misconceptions of high school students in genetics content. Genetics, 178(3), 1157–1168. https://doi.org/10.1534/genetics.107.084194.

    Article  Google Scholar 

  • Squire, K. D., & Jan, M. (2007). Mad city mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), 5–29.

    Article  Google Scholar 

  • Steinkuehler, C. A., & Duncan, S. (2008). Scientific habits of mind in virtual worlds. Journal of Science Education and Technology, 17(6), 530–543. https://doi.org/10.1007/s10956-008-9120-8.

    Article  Google Scholar 

  • Tsui, C., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. International Journal of Science Education, 32(8), 1073–1098. https://doi.org/10.1080/09500690902951429.

    Article  Google Scholar 

  • Wilensky, U., & Resnick, M. (1999). Thinking in levels: A dynamic systems approach to making sense of the world. Journal of Science Education and Technology, 8(1), 3–19.

    Article  Google Scholar 

Download references

Acknowledgement

We would like to thank the members of the Radix team at MIT who made this game and research possible, including Angie Tung, Susannah Gordon-Messer, and Jody Clarke-Midura.

Funding

This research was funded by the Bill and Melinda Gates Foundation and the Ministry of Science and Technology (MOST), Taiwan, under grant contract no. 104-2918-I-018-008 and 105-2511-S-018-015-MY5.

Author information

Authors and Affiliations

  1. Massachusetts Institute of Technology, MIT Education Arcade, Cambridge, MA, 02139, USA

    Louisa Rosenheck & Eric Klopfer

  2. Department of Biology, National Changhua University of Education, No. 1, Jin-De Road, Changhua, 500, Taiwan

    Meng-Tzu Cheng

  3. Eli Lilly and Company, Indianapolis, IN, 46285, USA

    Chen-Yen Lin

Authors
  1. Louisa Rosenheck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng-Tzu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen-Yen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric Klopfer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng-Tzu Cheng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

This study was carried out with institutional review board (IRB) approval and in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

All participants were well informed about the research objectives and contents and were assured about the privacy. None of the personal identifiers was reported in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rosenheck, L., Cheng, MT., Lin, CY. et al. Approaches to illuminate content-specific gameplay decisions using open-ended game data. Education Tech Research Dev 69, 1135–1154 (2021). https://doi.org/10.1007/s11423-021-09989-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-021-09989-0

Keywords

  • Games
  • Science learning
  • Gameplay patterns
  • Game learning analytics