Abstract
This study intends to use data visualization to examine learners’ behaviors in a 3D immersive serious game for middle school science to understand how the players interact with various features to solve the central problem. The analysis combined game log data with measures of in-game performance and learners’ goal orientations. The findings indicated students in the high performance and mastery-oriented groups tended to use the tools more appropriately relative to the stage they were at in the problem-solving process, and more productively than students in low performance groups. The use of data visualization with log data in combination with more traditional measures shows visualization as a promising technique in analytics with multiple data sets that can facilitate the interpretation of the relationships among data points at no cost to the complexity of the data. Design implications and future applications of serious games analytics and data visualization to the serious game are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abt, C. C. (1970). Serious games. New York: The Viking Press.
Ames, C. (1992). Achievement goals and classroom motivational climate. In J. Meece & D. Schunk (Eds.), Students’ perceptions in the classroom (pp. 327–348). Hillsdale, NJ: Erlbaum.
Andersen, E., Liu, Y. E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. In Proceedings from The Fifth International Conference on the Foundations of Digital Games, Pacific Grove, CA (pp. 1–8). doi:10.1145/1822348.1822349.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.
Balzer, W. K., Doherty, M. E., & O’Connor, R. (1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106(3), 410.
Barab, S. A., Gresalfi, M., & Ingram-Goble, A. (2010). Transformational play using games to position person, content, and context. Educational Researcher, 39(7), 525–536.
Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. New York: W.H. Freeman and Company.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178. doi:10.1207/s15327809jls0202_2.
Cobb, P., Confrey, J., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. doi:10.3102/0013189X032001009.
Dede, C. (2014, May 6). Data visualizations in immersive, authentic simulations for learning [Flash slides]. Retrieved from http://www.edvis.org/tuesday-presentations/
Dixit, P. N., & Youngblood, G. M. (2008). Understanding playtest data through visual data mining in interactive 3d environments. In Proceedings from 12th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia and Serious Games (CGAMES) (pp. 34–42).
Djaouti, D., Alvarez, J., Jessel, J. P., & Rampnoux, O. (2011). Origins of serious games. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious games and edutainment applications (pp. 25–43). Berlin, Germany: Springer.
Drachen, A., & Canossa, A. (2009). Towards gameplay analysis via gameplay metrics. In Proceedings from the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era (pp. 202–209). ACM. doi:10.1145/1621841.1621878.
Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040–1048.
Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational Psychologist, 34, 169–189.
Elliot, A. J., & Harackiewicz, J. M. (1996). Approach and avoidance achievement goals and intrinsic motivation: A mediational analysis. Journal of Personality and Social Psychology, 70, 461–475.
Garzotto, F. (2007). Investigating the educational effectiveness of multiplayer online games for children. In Proceedings from the 6th International Conference on Interaction Design and Children, Aalborg, Denmark (pp. 29–36). doi:10.1145/1297277.1297284.
Holcomb, J., & Mitchell, A. (2014, March). The revenue picture for American journalism and how it is changing. Retrieved from http://www.journalism.org/2014/03/26/the-revenue-picture-for-american-journalism-and-how-it-is-changing/
Hsieh, P., Cho, Y., Liu, M., & Schallert, D. (2008). Examining the interplay between middle school students’ achievement goals and self-efficacy in a technology-enhanced learning environment. American Secondary Education, 36(3), 33–50.
Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H. (2013). NMC horizon report: 2013 Higher Education Edition. Austin, TX: The New Media Consortium.
Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2014). NMC horizon report: 2014 Higher Education Edition. Austin, TX: The New Media Consortium.
Lajoie, S. P. (1993). Computer environments as cognitive tools for enhancing learning. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 261–288). Hillsdale, NJ: Lawrence Erlbaum Associates.
Linek, S. B., Marte, B., & Albert, D. (2008). The differential use and effective combination of questionnaires and logfiles. In Computer-Based Knowledge & Skill Assessment and Feedback in Learning Settings (CAF), Proceedings from The International Conference on Interactive Computer Aided Learning (ICL), Villach, Austria.
Linek, S. B., Öttl, G., & Albert, D. (2010). Non-invasive data tracking in educational games: Combination of logfiles and natural language processing. In L. G. Chova, D. M. Belenguer (Eds.), Proceedings from INTED 2010: International Technology, Education and Development Conference, Spain, Valenica.
List, J., & Bryant, B. (2014, March). Using Minecraft to encourage critical engagement of geography concepts. In Society for Information Technology & Teacher Education International [Conference Proceedings] (pp. 2384–2388). Jacksonville, FL.
Liu, M., & Bera, S. (2005). An analysis of cognitive tool use patterns in a hypermedia learning environment. Educational Technology Research and Development, 53(1), 5–21. doi:10.1007/BF02504854.
Liu, M., Bera, S., Corliss, S., Svinicki, M., & Beth, A. (2004). Understanding the connection between cognitive tool use and cognitive processes as used by sixth graders in a problem-based hypermedia learning environment. Journal of Educational Computing Research, 31(3), 309–334.
Liu, M., Horton, L. R., Corliss, S. B., Svinicki, M. D., Bogard, T., Kim, J., et al. (2009). Students’ problem solving as mediated by their cognitive tool use: A study of tool use patterns. Journal of Educational Computing Research, 40(1), 111–139.
Liu, M., Horton, L., Kang, J., Kimmons, R., & Lee, J. (2013). Using a ludic simulation to make learning of middle school space science fun. The International Journal of Gaming and Computer-Mediated Simulations, 5(1), 66–86. doi:10.4018/jgcms.2013010105.
Liu, M., Horton, L., Toprac, P., & Yuen, T. T. (2012). Examining the design of media-rich cognitive tools as scaffolds in a multimedia problem-based learning environment. In Educational media and technology yearbook (pp. 113–125). New York: Springer.
Liu, M., Wivagg, J., Geurtz, R., Lee, S.-T., & Chang, H. M. (2012). Examining how middle school science teachers implement a multimedia-enriched problem-based learning environment. Interdisciplinary Journal of Problem-Based Learning, 6(2), 46–84.
Loh, C. S. (2008). Designing online games assessment as “Information Trails”. In V. Sugumaran (Ed.), Intelligent information technologies: Concepts, methodologies, tools, and applications (pp. 553–574). Hershey, PA: Information Science Reference. doi:10.4018/978-1-59904-941-0.ch032.
Loh, C. S. (2011). Using in situ data collection to improve the impact and return of investment of game-based learning. In Old Meets New: Media in Education—Proceedings of the 61st International Council for Educational Media and the XIII International Symposium on Computers in Education (ICEM & SIIE’2011) Joint Conference (pp. 801–811). doi: 10.4018/jvple.2013010101.
Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599. doi:10.1016/j.compedu.2009.09.008.
Middleton, M. J., & Midgley, C. (1997). Avoiding the demonstration of lack of ability: An underexplored aspect of goal theory. Journal of Educational Psychology, 89, 710–718.
Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., et al. (2000). Patterns of adaptive learning scales (PALS). Ann Arbor, MI: University of Michigan.
Midgley, C., & Urdan, T. (1995). Predictors of middle school students’ use of self-handicapping strategies. The Journal of Early Adolescence, 15, 389–411.
Milam, D., & El Nasr, M. S. (2010, July). Design patterns to guide player movement in 3D games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games (pp. 37–42). ACM. doi:10.1145/1836135.1836141.
Pajares, F., Britner, S., & Valiante, G. (2000). Relation between achievement goals and self-beliefs of middle school students in writing and science. Contemporary Educational Psychology, 25, 406–422.
Reese, D. D., Tabachnick, B. G., & Kosko, R. E. (2013). Video game learning dynamics: Actionable measures of multidimensional learning trajectories. British Journal of Educational Technology. doi:10.1111/bjet.12128.
Rideout, V. J., Foehr, U. G., & Roberts, D.F. (2010, January). Generation M2: Media in the lives of 8- to 18-year-olds. Kaiser Family Foundation. Retrieved from http://kff.org/other/poll-finding/report-generation-m2-media-in-the-lives/
Rieber, L. (1996). Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research and Development, 44(2), 43–58. doi:10.1007/BF02300540.
Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(6), 601–618. doi:10.1109/TSMCC.2010.2053532.
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27. doi:10.1002/widm.1075.
Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j.compedu.2007.05.016.
Rosenbaum, E., Klopfer, E., & Perry, J. (2007). On location learning: Authentic applied science with networked augmented realities. Journal of Science Education and Technology, 16(1), 31–45. doi:10.1007/sl0956-006-9036-0.
Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press.
Sawyer, B., & Smith, P. (2008). Serious games taxonomy. [PDF document]. Retrieved from http://www.dmill.com/presentations/serious-games-taxonomy-2008.pdf
Scarlatos, L. L., & Scarlatos, T. (2010). Visualizations for the assessment of learning in computer games. In 7th International Conference & Expo on Emerging Technologies for a Smarter World (CEWIT 2010), September 27–29 2010, Incheon, Korea.
Serrano, A., Marchiori, E. J., del Blanco, A., Torrente, J., & Fernández-Manjón, B. (2012, April). A framework to improve evaluation in educational games. In Proceedings from Global Engineering Education Conference (EDUCON), 2012 IEEE (pp. 1–8). IEEE. doi:10.1109/EDUCON.2012.6201154.
Squire, K. D. (2004). Review. Simulation & Gaming, 35(1), 135–140. doi:10.1177/1046878103255490.
Squire, K. D., & Jan, M. (2007). Mad City Mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), 5–29. doi:10.1007/s10956-006-9037-z.
Sweetser, P., & Wyeth, P. (2005). GameFlow: A model for evaluating player enjoyment in games. Computers in Entertainment, 3(3), 3–3.
Tanes, Z., & Cemalcilar, Z. (2010). Learning from SimCity: An empirical study of Turkish adolescents. Journal of Adolescence, 33(5), 731–739.
U.S. Department of Education, Office of Educational Technology (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington, DC.
van Barneveld, A., Arnold, K. E., & Campbell, J. P. (2012). Analytics in higher education: Establishing a common language. EDUCAUSE Learning Initiative. Retrieved from https://qa.itap.purdue.edu/learning/docs/research/ELI3026.pdf
Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi:10.1016/j.entcom.2013.02.002.
Acknowledgments
We would like to acknowledge the help by Damilola Shonaike in creating the image in Figure 14 as part of her 2014 summer CERT REU internship program. We also appreciate the help from Divya Thakur and Kelly Gaither from the Texas Advanced Computing Center at the University of Texas at Austin in exploring the use of the Processing language to create visualizations in the specific game environment.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Liu, M., Kang, J., Lee, J., Winzeler, E., Liu, S. (2015). Examining Through Visualization What Tools Learners Access as They Play a Serious Game for Middle School Science. In: Loh, C., Sheng, Y., Ifenthaler, D. (eds) Serious Games Analytics. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-05834-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-05834-4_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05833-7
Online ISBN: 978-3-319-05834-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)