Examining Through Visualization What Tools Learners Access as They Play a Serious Game for Middle School Science
This study intends to use data visualization to examine learners’ behaviors in a 3D immersive serious game for middle school science to understand how the players interact with various features to solve the central problem. The analysis combined game log data with measures of in-game performance and learners’ goal orientations. The findings indicated students in the high performance and mastery-oriented groups tended to use the tools more appropriately relative to the stage they were at in the problem-solving process, and more productively than students in low performance groups. The use of data visualization with log data in combination with more traditional measures shows visualization as a promising technique in analytics with multiple data sets that can facilitate the interpretation of the relationships among data points at no cost to the complexity of the data. Design implications and future applications of serious games analytics and data visualization to the serious game are discussed.
KeywordsSerious games Problem-based learning Middle school science Learner behaviors Goal orientation
We would like to acknowledge the help by Damilola Shonaike in creating the image in Figure 14 as part of her 2014 summer CERT REU internship program. We also appreciate the help from Divya Thakur and Kelly Gaither from the Texas Advanced Computing Center at the University of Texas at Austin in exploring the use of the Processing language to create visualizations in the specific game environment.
- Abt, C. C. (1970). Serious games. New York: The Viking Press.Google Scholar
- Ames, C. (1992). Achievement goals and classroom motivational climate. In J. Meece & D. Schunk (Eds.), Students’ perceptions in the classroom (pp. 327–348). Hillsdale, NJ: Erlbaum.Google Scholar
- Andersen, E., Liu, Y. E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. In Proceedings from The Fifth International Conference on the Foundations of Digital Games, Pacific Grove, CA (pp. 1–8). doi: 10.1145/1822348.1822349.
- Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.Google Scholar
- Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.Google Scholar
- Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. New York: W.H. Freeman and Company.Google Scholar
- Dede, C. (2014, May 6). Data visualizations in immersive, authentic simulations for learning [Flash slides]. Retrieved from http://www.edvis.org/tuesday-presentations/
- Dixit, P. N., & Youngblood, G. M. (2008). Understanding playtest data through visual data mining in interactive 3d environments. In Proceedings from 12th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia and Serious Games (CGAMES) (pp. 34–42).Google Scholar
- Drachen, A., & Canossa, A. (2009). Towards gameplay analysis via gameplay metrics. In Proceedings from the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era (pp. 202–209). ACM. doi: 10.1145/1621841.1621878.
- Garzotto, F. (2007). Investigating the educational effectiveness of multiplayer online games for children. In Proceedings from the 6th International Conference on Interaction Design and Children, Aalborg, Denmark (pp. 29–36). doi: 10.1145/1297277.1297284.
- Holcomb, J., & Mitchell, A. (2014, March). The revenue picture for American journalism and how it is changing. Retrieved from http://www.journalism.org/2014/03/26/the-revenue-picture-for-american-journalism-and-how-it-is-changing/
- Hsieh, P., Cho, Y., Liu, M., & Schallert, D. (2008). Examining the interplay between middle school students’ achievement goals and self-efficacy in a technology-enhanced learning environment. American Secondary Education, 36(3), 33–50.Google Scholar
- Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H. (2013). NMC horizon report: 2013 Higher Education Edition. Austin, TX: The New Media Consortium.Google Scholar
- Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2014). NMC horizon report: 2014 Higher Education Edition. Austin, TX: The New Media Consortium.Google Scholar
- Lajoie, S. P. (1993). Computer environments as cognitive tools for enhancing learning. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 261–288). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
- Linek, S. B., Marte, B., & Albert, D. (2008). The differential use and effective combination of questionnaires and logfiles. In Computer-Based Knowledge & Skill Assessment and Feedback in Learning Settings (CAF), Proceedings from The International Conference on Interactive Computer Aided Learning (ICL), Villach, Austria.Google Scholar
- Linek, S. B., Öttl, G., & Albert, D. (2010). Non-invasive data tracking in educational games: Combination of logfiles and natural language processing. In L. G. Chova, D. M. Belenguer (Eds.), Proceedings from INTED 2010: International Technology, Education and Development Conference, Spain, Valenica.Google Scholar
- List, J., & Bryant, B. (2014, March). Using Minecraft to encourage critical engagement of geography concepts. In Society for Information Technology & Teacher Education International [Conference Proceedings] (pp. 2384–2388). Jacksonville, FL.Google Scholar
- Liu, M., Bera, S., Corliss, S., Svinicki, M., & Beth, A. (2004). Understanding the connection between cognitive tool use and cognitive processes as used by sixth graders in a problem-based hypermedia learning environment. Journal of Educational Computing Research, 31(3), 309–334.CrossRefGoogle Scholar
- Loh, C. S. (2008). Designing online games assessment as “Information Trails”. In V. Sugumaran (Ed.), Intelligent information technologies: Concepts, methodologies, tools, and applications (pp. 553–574). Hershey, PA: Information Science Reference. doi: 10.4018/978-1-59904-941-0.ch032.CrossRefGoogle Scholar
- Loh, C. S. (2011). Using in situ data collection to improve the impact and return of investment of game-based learning. In Old Meets New: Media in Education—Proceedings of the 61st International Council for Educational Media and the XIII International Symposium on Computers in Education (ICEM & SIIE’2011) Joint Conference (pp. 801–811). doi: 10.4018/jvple.2013010101.
- Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., et al. (2000). Patterns of adaptive learning scales (PALS). Ann Arbor, MI: University of Michigan.Google Scholar
- Milam, D., & El Nasr, M. S. (2010, July). Design patterns to guide player movement in 3D games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games (pp. 37–42). ACM. doi: 10.1145/1836135.1836141.
- Rideout, V. J., Foehr, U. G., & Roberts, D.F. (2010, January). Generation M2: Media in the lives of 8- to 18-year-olds. Kaiser Family Foundation. Retrieved from http://kff.org/other/poll-finding/report-generation-m2-media-in-the-lives/
- Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press.Google Scholar
- Sawyer, B., & Smith, P. (2008). Serious games taxonomy. [PDF document]. Retrieved from http://www.dmill.com/presentations/serious-games-taxonomy-2008.pdf
- Scarlatos, L. L., & Scarlatos, T. (2010). Visualizations for the assessment of learning in computer games. In 7th International Conference & Expo on Emerging Technologies for a Smarter World (CEWIT 2010), September 27–29 2010, Incheon, Korea.Google Scholar
- Serrano, A., Marchiori, E. J., del Blanco, A., Torrente, J., & Fernández-Manjón, B. (2012, April). A framework to improve evaluation in educational games. In Proceedings from Global Engineering Education Conference (EDUCON), 2012 IEEE (pp. 1–8). IEEE. doi: 10.1109/EDUCON.2012.6201154.
- U.S. Department of Education, Office of Educational Technology (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington, DC.Google Scholar
- van Barneveld, A., Arnold, K. E., & Campbell, J. P. (2012). Analytics in higher education: Establishing a common language. EDUCAUSE Learning Initiative. Retrieved from https://qa.itap.purdue.edu/learning/docs/research/ELI3026.pdf