Advertisement

Comparative Visualization of Player Behavior for Serious Game Analytics

  • Günter WallnerEmail author
  • Simone Kriglstein
Part of the Advances in Game-Based Learning book series (AGBL)

Abstract

Telemetry opens new possibilities for the assessment of serious games through the continuous, unobtrusive, monitoring of in-game behavior. Data obtained through telemetry thus not only contains information about the outcomes but also about the intermediate processes. In this sense, telemetry data can be of value for various stakeholders of serious games, including developers, educators, and learners themselves to increase the effectiveness of the intervention. In doing so, particular significance should be attached to differences among individuals and demographic groups in order to understand and better accommodate for these variations. However, the large amounts of data gathered via telemetry can make it challenging to derive meaningful information from it. Visualizations can support this process by providing a means to explore, to compare, and to draw insights from the data sets. In this chapter, we discuss three common visual design strategies that facilitate comparative data analysis. Several examples, drawn from the game-based learning literature and related areas as well as two detailed case studies are used to illustrate how these strategies can be leveraged in the context of serious game analytics.

Keywords

Game telemetry Player behavior Visualization Visual comparison 

Notes

Acknowledgments

We would like to thank Helmut Hlavacs from the Entertainment Computing group of the University of Vienna for his permission to use the telemetry data from Internet Hero for examples in this chapter. Internet Hero was funded by netidee (www.netidee.at, project number 326).

References

  1. Andersen, E., Liu, Y.-E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. In Proceedings of the 5th International Conference on the Foundations of Digital Games (pp. 1–8). doi: 10.1145/1822348.1822349.
  2. Andrews, K., Wohlfahrt, M., & Wurzinger, G. (2009). Visual graph comparison. In Proceedings of the 13th International Conference Information Visualisation (pp. 62–67). doi: 10.1109/IV.2009.108.
  3. Anolli, L., & Confalonieri, L. (2011). Learning, dynamic assessment and serious games. In A. Méndez-Vilas (Ed.), Education in a technological world: Communicating current and emerging research and technological efforts (pp. 279–287). Badajoz, Spain: Formatex.Google Scholar
  4. Archambault, D., Purchase, H. C., & Pinaud, B. (2011). Animation, small multiples, and the effect of mental map preservation in dynamic graphs. IEEE Transactions on Visualization and Computer Graphics, 17(4), 539–552. doi: 10.1109/TVCG.2010.78.CrossRefGoogle Scholar
  5. Arthur, W., Jr., Strong, M. H., Jordan, J. A., Williamson, J. E., Shebilske, W. L., & Regian, J. W. (1995). Visual attention: Individual differences in training and predicting complex task performance. Acta Psychologica, 88(1), 3–23. doi: 10.1016/0001-6918(94)E0055-K.CrossRefGoogle Scholar
  6. Beck, F., Burch, M., Diehl, S., & Weiskopf, D. (2014). The state of the art in visualizing dynamic graphs. In EuroVis—STARs (pp. 83–103). doi: 10.2312/eurovisstar.20141174.
  7. Beck, F., Burch, M., & Weiskopf, D. (2013). Visual comparison of time-varying athletes’ performance. In Proceedings of the 1st Workshop on Sports Data Visualization.Google Scholar
  8. Becker, K., & Parker, J. R. (2011). The guide to computer simulations and games. Indianapolis, IN: Wiley.Google Scholar
  9. Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction, 2013. doi:10.1155/2013/136864.Google Scholar
  10. Boyandin, I., Bertini, E., & Lalanne, D. (2012). A qualitative study on the exploration of temporal changes in flow maps with animation and small-multiples. Computer Graphics Forum, 31(3pt2), 1005–1014. doi: 10.1111/j.1467-8659.2012.03093.x.CrossRefGoogle Scholar
  11. Brandes, U., & Corman, S. R. (2003). Visual unrolling of network evolution and the analysis of dynamic discourse. Information Visualization, 2(1), 40–50. doi: 10.1057/palgrave.ivs.9500037.CrossRefGoogle Scholar
  12. Brandes, U., Dwyer, T., & Schreiber, F. (2004). Visualizing related metabolic pathways in two and a half dimensions. In G. Liotta (Ed.), Graph drawing (pp. 111–122). Berlin: Springer.CrossRefGoogle Scholar
  13. Buendía-García, F., García-Martínez, S., Navarrete-Ibañez, E. M., & Jesús, M. (2013). Designing serious games for getting transferable skills in training settings. Interaction Design and Architecture(s), 19, 47–62.Google Scholar
  14. Butler, E., & Banerjee, R. (2014). Visualizing progressions for education and game design. Retrieved from http://cse512-14w.github.io/fp-edbutler-piscean/final/paper-edbutler-piscean.pdf
  15. Dawson, S. (2010). “Seeing” the learning community: An exploration of the development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), 736–752. doi: 10.1111/j.1467-8535.2009.00970.x.CrossRefGoogle Scholar
  16. de Freitas, S., & Jarvis, S. (2006). A framework for developing serious games to meet learner needs. In Proceedings of the Interservice/Industry Training, Simulation & Education Conference.Google Scholar
  17. de Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & Education, 46(3), 249–264. doi: 10.1016/j.compedu.2005.11.007.CrossRefGoogle Scholar
  18. Desmarais, M. C., & Lemieux, F. (2013). Clustering and visualizing study state sequences. In Proceedings of 6th International Conference on Educational Data Mining (pp. 224–227).Google Scholar
  19. Drachen, A., & Canossa, A. (2011). Evaluating motion: Spatial user behaviour in virtual environments. International Journal of Arts and Technology, 4(3), 294–314. doi: 10.1504/IJART.2011.041483.CrossRefGoogle Scholar
  20. Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 9–17). doi: 10.1145/2090116.2090118.
  21. Dwyer, T., Hong, S.-H., Koschützki, D., Schreiber, F., & Xu, K. (2006). Visual analysis of network centralities. In Proceedings of the Asia-Pacific Symposium on Information Visualisation (pp. 189–197).Google Scholar
  22. Erten, C., Kobourov, S. G., Le, V., & Navabi, A. (2003). Simultaneous graph drawing: Layout algorithms and visualization schemes. In Proceedings of the 11th Symposium on Graph Drawing (pp. 437–449).Google Scholar
  23. Fardinpour, A., Reiners, T., & Dreher, H. (2013). Action-based learning assessment method (ALAM) in virtual training environments. In Proceedings of the 30th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 267–277).Google Scholar
  24. Federico, P., Aigner, W., Miksch, S., Windhager, F., & Zenk, L. (2011). A visual analytics approach to dynamic social networks. In Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies (pp. 47:1–47:8). doi: 10.1145/2024288.2024344.
  25. Ferster, B., & Shneiderman, B. (2012). Interactive visualization: Insight through inquiry. Cambridge, MA: MIT Press.Google Scholar
  26. Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. doi: 10.1007/s11528-014-0822-x.CrossRefGoogle Scholar
  27. Gelderblom, H., & Kotzé, P. (2009). Ten design lessons from the literature on child development and children’s use of technology. In Proceedings of the 8th International Conference on Interaction Design and Children (pp. 52–60). doi: 10.1145/1551788.1551798.
  28. Gleicher, M., Albers, D., Walker, R., Jusufi, I., Hansen, C. D., & Roberts, J. C. (2011). Visual comparison for information visualization. Information Visualization, 10(4), 289–309. doi: 10.1177/1473871611416549.CrossRefGoogle Scholar
  29. Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for awareness and self-reflection. In CHI’12 Extended Abstracts on Human Factors in Computing Systems (pp. 869–884). doi: 10.1145/2212776.2212860.
  30. Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010). Visualizing activities for self-reflection and awareness. In X. Luo, M. Spaniol, L. Wang, Q. Li, W. Nejdl, & W. Zhang (Eds.), Advances in web-based learning—ICWL 2010 (pp. 91–100). Berlin: Springer.CrossRefGoogle Scholar
  31. Graham, M., & Kennedy, J. (2010). A survey of multiple tree visualisation. Information Visualization, 9(4), 235–252. doi: 10.1057/ivs.2009.29.CrossRefGoogle Scholar
  32. Griffin, A. L., MacEachren, A. M., Hardisty, F., Steiner, E., & Li, B. (2006). A comparison of animated maps with static small-multiple maps for visually identifying space-time clusters. Annals of the Association of American Geographers, 96(4), 740–753. doi: 10.1111/j.1467-8306.2006.00514.x.CrossRefGoogle Scholar
  33. Guerra-Gómez, J. A., Buck-Coleman, A., Pack, M. L., Plaisant, C., & Shneiderman, B. (2013). TreeVersity: Interactive visualizations for comparing hierarchical data sets. Transportation Research Record: Journal of the Transportation Research Board, 2392, 48–58. doi: 10.3141/2392-06.CrossRefGoogle Scholar
  34. Hartmann, T., & Klimmt, C. (2006). Gender and computer games: Exploring females’ dislikes. Journal of Computer-Mediated Communication, 11(4), 910–931. doi: 10.1111/j.1083-6101.2006.00301.x.CrossRefGoogle Scholar
  35. Heeter, C., Lee, Y.-H., Magerko, B., & Medler, B. (2011). Impacts of forced serious game play on vulnerable subgroups. International Journal of Gaming and Computer-Mediated Simulations, 3(3), 34–53. doi: 10.4018/jgcms.2011070103.CrossRefGoogle Scholar
  36. Holten, D., & van Wijk, J. J. (2008). Visual comparison of hierarchically organized data. In Proceedings of the 10th Joint Eurographics/IEEE—VGTC Conference on Visualization (pp. 759–766). doi: 10.1111/j.1467-8659.2008.01205.x.
  37. Houghton, S. (2011). Balance and flow maps. Retrieved April, 2015, from http://www.gamasutra.com/view/news/125213/Opinion_Balance_and_Flow_Maps.php.
  38. Inselberg, A. (1985). The plane with parallel coordinates. The Visual Computer, 1(2), 69–91. doi: 10.1007/BF01898350.CrossRefGoogle Scholar
  39. Javed, W., & Elmqvist, N. (2012). Exploring the design space of composite visualization. In Proceedings of the IEEE Pacific Visualization Symposium (pp. 1–8). doi: 10.1109/PacificVis.2012.6183556.
  40. Kayali, F., Wallner, G., Kriglstein, S., Bauer, G., Martinek, D., Hlavacs, H., et al. (2014). A case study of a learning game about the Internet. In S. Göbel & J. Wiemeyer (Eds.), Games for training, education, health and sports (pp. 47–58). Cham, Switzerland: Springer.CrossRefGoogle Scholar
  41. Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment. Journal of Science Education and Technology, 16(1), 99–111. doi: 10.1007/s10956-006-9038-y.CrossRefGoogle Scholar
  42. Kiili, K., Ketamo, H., & Kickmeier-Rust, M. D. (2014). Evaluating the usefulness of eye tracking in game-based learning. International Journal of Serious Games, 1(2), 51–65.CrossRefGoogle Scholar
  43. Kirk, A. (2012). Data visualization: A successful design process. Birmingham, England: Packt Publishing.Google Scholar
  44. Klerkx, J., Verbert, K., & Duval, E. (2014). Enhancing learning with visualization techniques. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 791–807). New York: Springer.CrossRefGoogle Scholar
  45. Kriglstein, S., Pohl, M., & Smuc, M. (2014). Pep up your time machine: Recommendations for the design of information visualizations of time-dependent data. In W. Huang (Ed.), Handbook of human centric visualization (pp. 203–225). New York: Springer.CrossRefGoogle Scholar
  46. Kriglstein, S., Wallner, G., & Rinderle-Ma, S. (2013). A visualization approach for difference analysis of process models and instance traffic. In Proceedings of the 11th International Conference on Business Process Management (pp. 219–226). doi: 10.1007/978-3-642-40176-3_18.
  47. Kriz, W. C., & Hense, J. U. (2006). Theory-oriented evaluation for the design of and research in gaming and simulation. Simulation & Gaming, 37(2), 268–283. doi: 10.1177/1046878106287950.CrossRefGoogle Scholar
  48. Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage.Google Scholar
  49. Linek, S. B., Öttl, G., & Albert, D. (2010). Non-invasive data tracking in educational games: Combination of logfiles and natural language processing. In Proceeding of the International Technology, Education and Development Conference (pp. 2977–2988).Google Scholar
  50. Liu, Y.-E., Andersen, E., Snider, R., Cooper, S., & Popović, Z. (2011). Feature-based projections for effective playtrace analysis. In Proceedings of the 6th International Conference on Foundations of Digital Games (pp. 69–76). doi: 10.1145/2159365.2159375.
  51. Loh, C. S. (2012). Information Trails: In-process assessment of game-based learning. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning (pp. 123–144). New York: Springer.CrossRefGoogle Scholar
  52. Loh, C. S., & Sheng, Y. (2013). Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics. Education and Information Technologies, 20, 5–19. doi: 10.1007/s10639-013-9263-y.CrossRefGoogle Scholar
  53. Magerko, B., Heeter, C., & Medler, B. (2010). Different strokes for different folks: Tapping into the hidden potential of serious games. In R. Van Eck (Ed.), Gaming and cognition: Theories and practice from the learning sciences (pp. 255–280). Hershey, PA: IGI Global.CrossRefGoogle Scholar
  54. Mehigan, T. J., Barry, M., Kehoe, A., & Pitt, I. (2011). Using eye tracking technology to identify visual and verbal learners. In Proceedings of the IEEE International Conference on Multimedia and Expo (pp. 1–6). doi: 10.1109/ICME.2011.6012036.
  55. Miller, J. R. (2007). Attribute blocks: Visualizing multiple continuously defined attributes. IEEE Computer Graphics and Applications, 27(3), 57–69. doi: 10.1109/MCG.2007.54.CrossRefGoogle Scholar
  56. Minović, M., & Milovanović, M. (2013). Real-time learning analytics in educational games. In Proceedings of the 1st International Conference on Technological Ecosystem for Enhancing Multiculturality (pp. 245–251). doi: 10.1145/2536536.2536574.
  57. Mislevy, R. J., & Riconscente, M. (2005). Evidence-centered assessment design: Layers, structures, and terminology (PADI Technical Report No. 9). Menlo Park, CA: SRI International.Google Scholar
  58. Moreno-Ger, P., Torrente, J., Hsieh, Y. G., & Lester, W. T. (2012). Usability testing for serious games: Making informed design decisions with user data. Advances in Human-Computer Interaction, 2012. doi:10.1155/2012/369637.Google Scholar
  59. Munzner, T., Guimbretière, F., Tasiran, S., Zhang, L., & Zhou, Y. (2003). TreeJuxtaposer: Scalable tree comparison using focus + context with guaranteed visibility. ACM Transactions on Graphics, 22(3), 453–462. doi: 10.1145/1201775.882291.CrossRefGoogle Scholar
  60. O’Rourke, E., Butler, E., Liu, Y.-E., Ballweber, C., & Popović, Z. (2013). The effects of age on player behavior in educational games. In Proceedings of the 8th International Conference on the Foundations of Digital Games (pp. 158–165).Google Scholar
  61. Olsen, T., Procci, K., & Bowers, C. (2011). Serious games usability testing: How to ensure proper usability, playability, and effectiveness. In A. Marcus (Ed.), Design, user experience, and usability. Theory, methods, tools and practice (pp. 625–634). Berlin: Springer.CrossRefGoogle Scholar
  62. Peirce, K., & Edwards, E. D. (1988). Children’s construction of fantasy stories: Gender differences in conflict resolution strategies. Sex Roles, 18(7–8), 393–404. doi: 10.1007/BF00288391.CrossRefGoogle Scholar
  63. Perin, C., Vuillemot, R., & Fekete, J.-D. (2013). SoccerStories: A kick-off for visual soccer analysis. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2506–2515. doi: 10.1109/TVCG.2013.192.CrossRefGoogle Scholar
  64. Pileggi, H., Stolper, C. D., Boyle, J. M., & Stasko, J. T. (2012). SnapShot: Visualization to propel ice hockey analytics. IEEE Transactions on Visualization and Computer Graphics, 18(12), 2819–2828. doi: 10.1109/TVCG.2012.263.CrossRefGoogle Scholar
  65. Piringer, H., Pajer, S., Berger, W., & Teichmann, H. (2012). Comparative visual analysis of 2D function ensembles. Computer Graphics Forum, 31(3), 1195–1204. doi: 10.1111/j.1467-8659.2012.03112.x.CrossRefGoogle Scholar
  66. Plass, J. L., Homer, B. D., Kinzer, C. K., Chang, Y. K., Frye, J., Kaczetow, W., et al. (2013). Metrics in simulations and games for learning. In M. S. El-Nasr, A. Drachen, & A. Canossa (Eds.), Game analytics (pp. 697–729). London: Springer.CrossRefGoogle Scholar
  67. Ritsos, P. D., & Roberts, J. C. (2014). Towards more visual analytics in learning analytics. In Proceedings of the 5th EuroVis Workshop on Visual Analytics (pp. 61–65). doi: 10.2312/eurova.20141147.
  68. Roberts, J. C. (2005). Exploratory visualization with multiple linked views. In J. Dykes, A. M. MacEachren, & M.-J. Kraak (Eds.), Exploring geovisualization (pp. 159–180). Oxford, England: Elsevier.CrossRefGoogle Scholar
  69. Robertson, G., Fernandez, R., Fisher, D., Lee, B., & Stasko, J. (2008). Effectiveness of animation in trend visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1325–1332. doi: 10.1109/TVCG.2008.125.CrossRefGoogle Scholar
  70. Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2010). Individual differences in gameplay and learning: A narrative-centered learning perspective. In Proceedings of the 5th International Conference on the Foundations of Digital Games (pp. 171–178). doi: 10.1145/1822348.1822371.
  71. Scarlatos, L. L., & Scarlatos, T. (2010). Visualizations for the assessment of learning in computer games. In Proceedings of the 7th International Conference & Expo on Emerging Technologies for a Smarter World.Google Scholar
  72. Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., & Fernández-Manjón, B. (2012). Tracing a little for big improvements: Application of learning analytics and videogames for student assessment. Procedia Computer Science, 15, 203–209. doi: 10.1016/j.procs.2012.10.072.CrossRefGoogle Scholar
  73. Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14(1), 47–69. doi: 10.1023/A:1013180410169.CrossRefGoogle Scholar
  74. Shute, V., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfield, M. J. Cody, & P. Vorderer (Eds.), The social science of serious games: Theories and applications (pp. 295–321). New York: Routledge.Google Scholar
  75. Steiner, C. M., Kickmeier-Rust, M. D., & Albert, D. (2009). Little big difference: Gender aspects and gender-based adaptation in educational games. In M. Chang, R. Kuo, K. Kinshuk, G.-D. Chen, & M. Hirose (Eds.), Learning by playing. Game-based education system design and development (pp. 150–161). Berlin: Springer.CrossRefGoogle Scholar
  76. Stewart, C. A., Hart, D., Berry, D. K., Olsen, G. J., Wernert, E. A., & Fischer, W. (2001). Parallel implementation and performance of fastDNAml: A program for maximum likelihood phylogenetic inference. In Proceedings of the ACM/IEEE Conference on Supercomputing. doi: 10.1145/582034.582054.
  77. Tominski, C., Schulze-Wollgast, P., & Schumann, H. (2005). 3D information visualization for time dependent data on maps. In Proceedings of the nineth International Conference on Information Visualisation (pp. 175–181). doi: 10.1109/IV.2005.3.
  78. Tufte, E. R. (1990). Envisioning information. Cheshire, UK: Graphics Press.Google Scholar
  79. Urness, T., Interrante, V., Marusic, I., Longmire, E., & Ganapathisubramani, B. (2003). Effectively visualizing multi-valued flow data using color and texture. In Proceedings of the IEEE Visualization (pp. 115–121). doi: 10.1109/VISUAL.2003.1250362.
  80. van Wijk, J. J. (2005). The value of visualization. In Proceedings of the IEEE Visualization (pp. 79–86). doi: 10.1109/VISUAL.2005.1532781.
  81. Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Assche, F. V., Parra, G., et al. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. doi: 10.1007/s00779-013-0751-2.Google Scholar
  82. Wallner, G. (2013). Play-Graph: A methodology and visualization approach for the analysis of gameplay data. In Proceedings of the 8th International Conference on the Foundations of Digital Games (pp. 253–260).Google Scholar
  83. Wallner, G., & Kriglstein, S. (2012a). DOGeometry: Teaching geometry through play. In Proceedings of the 4th International Conference on Fun and Games (pp. 11–18). doi: 10.1145/2367616.2367618.
  84. Wallner, G., & Kriglstein, S. (2012b). A spatiotemporal visualization approach for the analysis of gameplay data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1115–1124). doi: 10.1145/2207676.2208558.
  85. Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi: 10.1016/j.entcom.2013.02.002.CrossRefGoogle Scholar
  86. Ward, M. O. (2002). A taxonomy of glyph placement strategies for multidimensional data visualization. Information Visualization, 1(3/4), 194–210. doi: 10.1057/palgrave.ivs.9500025.CrossRefGoogle Scholar
  87. Warren, S., Jones, G., & Lin, L. (2011). Usability and play testing. In L. Annetta & S. C. Bronack (Eds.), Serious educational game assessment (pp. 131–146). Rotterdam, The Netherlands: Sense.CrossRefGoogle Scholar
  88. Westera, W., Nadolski, R., Hummel, H. (2014). Serious gaming analytics: What students’ log files tell us about gaming and learning. International Journal of Serious Games, 1(2). doi:10.17083/ijsg.v1i2.9.Google Scholar
  89. Wouters, P., van der Spek, E. D., & Van Oostendorp, H. (2009). Current practices in serious game research: A review from a learning outcomes perspective. In T. M. Connolly, M. Stansfield, & L. Boyle (Eds.), Games-based learning advancements for multisensory human computer interfaces: Techniques and effective practices. Hershey, PA: IGI Global.Google Scholar
  90. Yau, N. (2011). Visualize this: The FlowingData guide to design, visualization, and statistics. Indianapolis, IN: Wiley.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.University of Applied Arts ViennaViennaAustria
  2. 2.Vienna University of TechnologyViennaAustria

Personalised recommendations