Skip to main content
SpringerLink
Log in

Information Trails: In-Process Assessment of Game-Based Learning

  • Chapter
  • First Online:
Assessment in Game-Based Learning

Abstract

Assessment is an issue that is important to educators because without it, there is no way of telling if learners have arrived at the intended destination. Conducting assessments with game-based learning requires new tools and research methodologies because traditional face-to-face techniques do not transfer well into the multi-user virtual environments found in game worlds. Formative assessment is more useful to instructors and learners because it provides multi-point feedback to help them self-reflect and improve on what they are doing. This chapter describes an “all-rounded” assessment system for game-based learning, which take into consideration the needs of multiple consumers (of information), namely, the Administrators, the Trainers or Educators, and the Learners. The assessment system made use of Information Trails ©—a viable assessment methodology to collect user-generated action data as the game-based learning is occurring (hence, an in-process assessment). The data collected are then remotely transmitted to an external data storage using telemetry, and displayed in real-time via a data visualization application known as Performance Tracing Report Assistant © (or PeTRA). The final online assessment report can be tailored differently according to individual needs of the Learners, Trainers, and Administrators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aldrich, C. (2009). The complete guide to simulations and serious games (p. 576). San Francisco, CA: Pfeiffer.

    Google Scholar 

  • Baker, R. S., Corbett, A. T., Roll, I., & Koedinger, K. R. (2008). Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, 18(3), 287–314.

    Article  Google Scholar 

  • BBC News. (2007). Computer game to boost key skills. Retrieved Jan 22, 2011 from http://news.bbc.co.uk/2/hi/uk_news/education/6254989.stm.

  • Berger, A. (2006). “Neverwinter Nights” in the classroom. UMN News. Retrieved Sept 9, 2011 from http://www1.umn.edu/news/features/2006/UR_83484_REGION1.html.

  • Black, P. J., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5(1), 1–73.

    Article  Google Scholar 

  • Black, P. J., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 1(1), 5–31.

    Article  Google Scholar 

  • Botturi, L., & Loh, C. S. (2008). Once upon a game: Rediscovering the roots of games in education. In C. T. Miller (Ed.), Games: Purpose and potential in education (pp. 1–22). New York, NY: Springer Science.

    Google Scholar 

  • Chen, S., & Michael, D. (2005). Proof of learning: Assessment in serious games. Gamasutra. Retrieved Jan 12, 2011 from http://www.gamasutra.com/features/20051019/chen_01.shtml.

  • Clark, R. E. (2007). Learning from serious games? Arguments, evidence and research suggestions. Educational Technology, 47, 56–59.

    Google Scholar 

  • Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimum Experience. New York, NY: Harper & Row.

    Google Scholar 

  • Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Dragon Age: Origins (2009). [Computer software]. Redwood City, CA: EA Games.

    Google Scholar 

  • Duffy, J. (1997). The role of virtual reality simulation in full scale joint military exercises (p. 17). Newport, RI: Naval War College.

    Google Scholar 

  • Gee, J. P. (2007). What video games have to teach us about learning and literacy (p. 256). New York, NY: Palgrave Macmillan.

    Google Scholar 

  • Gibson, D., Aldrich, C., & Prensky, M. (Eds.). (2006). Games and simulations in online learning: Research and development frameworks (p. 402). Hershey, PA: Information Science.

    Google Scholar 

  • Gorniak, P., & Roy, D. (2005). Speaking with your sidekick: Understanding situated speech in computer role playing games. In R. M. Young & J. E. Laird (Eds.), Proceedings of the first artificial intelligence and interactive digital entertainment conference (AIIDE 2005), June 1–5, 2005. Marina del Rey, CA: AAAI.

    Google Scholar 

  • Harrington, H. L., Meisels, S. J., McMahon, P., Dichtelmiller, M. L., & Jablon, J. R. (1997). Observing, documenting and assessing learning. Ann Arbor, MI: Rebus.

    Google Scholar 

  • Hays, R. T. (2005). The effectiveness of instructional games: A literature review and discussion (p. 63). Orlando, FL: Naval Air Warfare Center.

    Google Scholar 

  • Hattie, J. (1987). Identifying the salient facets of a model of student learning: A synthesis of meta analyses. International Journal of Educational Research, 11(2), 187–212.

    Google Scholar 

  • Ifenthaler, D., Eseryel, D., & Ge, X. (Eds.). (2012). Assessment in game-based learning: Foundations, innovations, and perspectives. New York, NY: Springer Science+Business Media New York.

    Google Scholar 

  • Joosten-ten Brinke, D., Gorissen, P., & Latour, I. (2005). Integrating assessment into e-learning courses. In R. Koper & C. Tattersall (Eds.), Learning design: A handbook on modelling and delivering networked education and training (pp. 185–202). Berlin: Springer.

    Google Scholar 

  • Kaplan-Rakowski, R., & Loh, C. S. (2010). Modding and rezzing in games and virtual environments for education. In Y. K. Baek (Ed.), Gaming for classroom-based learning: Digital role playing as a motivator of study (pp. 205–219). Hershey, PA: IGI-Global.

    Chapter  Google Scholar 

  • Kapp, K. M., & O’Driscoll, T. (2010). Learning in 3D: Adding a new dimension to enterprise learning and collaboration (p. 416). San Francisco, CA: Pfeiffer.

    Google Scholar 

  • Kebritchi, M. (2008). Effects of a computer game on mathematics achievement and class motivation: An experimental study. PhD Thesis. University of Central Florida, Orlando, Florida.

    Google Scholar 

  • Lee, H. (2008). Students’ perceptions of peer and self assessment in a higher education online collaborative learning environment. Unpublished Ph.D. dissertation, Graduate School of the University of Texas at Austin, Austin. Retrieved February 8, 2011 from http://www.lib.utexas.edu/etd/d/2008/leeh55399/leeh55399.pdf.

  • Levin, A. (2010). Simulator training flaws tied to airline crashes. USA Today. Retrieved March 1, 2011 from http://travel.usatoday.com/flights/2010-08-31-1Acockpits31_ST_N.htm.

  • Loh, C. S. (2006). Tracking an avatar: Designing data collection into online games. Paper presented at the Annual Conference of the Association for Educational Communications and Technology (AECT 2006). Dallas, TX.

    Google Scholar 

  • Loh, C. S. (2006b). Designing online games assessment as “Information Trails”. In D. Gibson, C. Aldrich, & M. Prensky (Eds.), Games and simulation in online learning: Research and development frameworks (pp. 323–348). Hershey, PA: Idea Group.

    Chapter  Google Scholar 

  • Loh, C. S. (2009). Researching and developing serious games as interactive learning instructions. International Journal of Gaming and Computer Mediated Simulations, 1(4), 1–19.

    Article  Google Scholar 

  • Loh, C. S., Anantachai, A., Byun, J. H., & Lenox, J. (2007). Assessing what players learned in serious games: In situ data collection, Information Trails, and quantitative analysis. In Q. Mehdi (Ed.), Proceedings of the computer games: AI, animation, mobile, educational & serious games conference (CGAMES 2007). Wolverhampton, UK: University of Wolverhampton.

    Google Scholar 

  • Loh, C. S., & Byun, J. H. (2009). Modding Neverwinter Nights into serious game. In D. Gibson & Y. K. Baek (Eds.), Digital Simulations for Improving Education: Learning Through Artificial Teaching Environments (pp. 408–426). Hershey, PA: IGI-Global.

    Chapter  Google Scholar 

  • Loh, C. S., & Li, I. H. (2010, Oct 10–13). Reducing re-training cost through on-demand, ad hoc assessment. Proceedings of the MODSIM world conference and expo 2010—21st century decision-making: the art of modeling & simulation, Hampton, VA: MODSIM World Conference & Exposition.

    Google Scholar 

  • Moreno-Ger, P., Burgos, D., Martínez-Ortiz, I., Sierra, J. L., & Fernández-Manjón, B. (2008). Educational game design for online education. Computers in Human Behavior, 24, 2530–2540.

    Article  Google Scholar 

  • Nacke, L., Grimshaw, M., & Lindley, C. (2010). More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers, 22(5), 336–343. doi:10.1016/j.intcom.2010.04.005.

    Article  Google Scholar 

  • Nacke, L., Lindley, C., & Stellmach, S. (2008). Log who’s playing: Psychophysiological game analysis made easy through event logging. In P. Markopoulos et al. (Eds.), Fun and games 2008, lecture notes of computer science (Vol. 5294, pp. 150–157). Heidelberg, Germany: Springer.

    Google Scholar 

  • Nacke, L., & Lindley, C. (2010). Affective ludology, flow and immersion in a first-person shooter: Measurement of player experience. Loading, 3(5). Retrieved March 12, 2011 from http://journals.sfu.ca/loading/index.php/loading/article/view/72/71

  • O’Neil, H. F., & Robertson, M. M. (1992). Simulations: Occupationally oriented. In M. C. Alkin (Ed.), Encyclopedia of Educational Research (6th ed., pp. 1216–1222). New York, NY: Macmillan.

    Google Scholar 

  • Prensky, M. (2001). Digital game-based learning. New York, NY: McGraw-Hill.

    Google Scholar 

  • Prensky, M. (2006). Don’t bother me mom—I’m learning! St. Paul, MN: Paragon House.

    Google Scholar 

  • Pruett, C. (2010, Sept). Hot failure: Tuning gameplay with simple player metrics. Game Developer Magazine. 19(9). San Francisco, CA: Think Services. Retrieved March 7, 2011 from http://gamasutra.com/view/feature/6155/hot_failure_tuning_gameplay_with_.php.

  • Reese, D. D. (2010). Introducing flowometer: A CyGaMEs assessment suite tool. In R. van Eck (Ed.), Gaming and cognition: Theories and practice from the learning sciences (pp. 227–254). Hershey, PA: IGI-Global.

    Chapter  Google Scholar 

  • Robert, L. G. (2003). Forewords. In G. D. Haertel & B. Means (Eds.), Evaluating educational technology: Effective research designs for improving learning (p. 290). New York, NY: Teachers College Press.

    Google Scholar 

  • Robertson, J., & Good, J. (2005). Story creation in virtual game worlds. Communications of the ACM, 48(1), 61–65.

    Article  Google Scholar 

  • Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. The Journal of Technology Learning and Assessment, 8(4), 3–41.

    Google Scholar 

  • Saridaki, M., Mourlas, C., Gouscos, D., & Meimaris, M. (2007). Digital games as a learning tool for children with cognitive disabilities: Literature review and some preliminary methodological and experimental results. Paper presented at the European Conference on Games Based Learning, Scotland

    Google Scholar 

  • Sasse, D. (2008). A framework for psychophysiological data acquisition in digital games. Unpublished Master thesis, Lund University, Lund. Retrieved March 12, 2011 from http://citeseerx.ist.psu.edu/viewdoc/download? doi: 10.1.1.140.5284&rep=rep1&type=pdf.

  • Shute, V. J., Masduki, I., Donmez, O., Dennen, V. P., Kim, Y.-J., Jeong, A. C., et al. (2010). Modeling, assessing, and supporting key competencies within game environments. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (Part 4, pp. 281–309). Boston, MA: Springer. Retrieved March 5, 2011, from http://www.springerlink.com/content/t5261055733n18h2/.

  • Shute, V. J., & Spector, J. M. (2008). SCORM 2.0 White paper: Stealth assessment in virtual worlds. Retrieved March 12, 2011, from http://www.adlnet.gov/technologies/evaluation/library/additionalresources/letsiwhitepapers/shute-stealthassessmentinvirtualworlds.pdf.

  • Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Cambridge, MA: B.F. Skinner Foundation.

    Google Scholar 

  • Stieger, I. (2008). Neverwinter Nights eXtender 4 (version 1.09) [software]. Starnberg, Germany: Stieger Hardware-und Softwareentwicklung. Retrieved April 12, 2012, from http://www.nwnx.org.

  • Telemetry. (n.d.). The American Heritage Stedman’s Medical Dictionary. Retrieved March 9, 2011, from http://dictionary.reference.com/browse/telemetry.

  • Tobias, S., & Fletcher, J. D. (2007). What research has to say about designing computer games for learning. Educational Technology, 47(5), 20–29.

    Google Scholar 

  • van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless…. Educause Review, 41(2), 16–30.

    Google Scholar 

  • Weil, S. A., Hussain, T. S., Brunyé, T. T., Ferguson, W., Sidman, J. G., Spahr, L.S., & Roberts, B. (2005). Assessing the potential of massive multi-player games as tools for military training. Proceedings of the interservice/industry training, simulation, and education conference, (I/ITSEC 2005). Orlando, FL: NTSA.

    Google Scholar 

  • Warren, R., & Sutton, J. (2008, April). Using a computer game for research on culture and team adaptability: Lessons learned from a NATO experiment. Proceedings of the NATO Research and Technology Organization (RTO) Human Factors and Medicine Panel HFM, 142 Symposium on Adaptability in Coalition Teamwork. Copenhagen, Denmark.

    Google Scholar 

  • Wilson, K. A., Bedwell, W. L., Lazzara, E. H., Salas, E., Burke, C. S., Estock, J. L., et al. (2008). Relationships between game attributes and learning outcomes: Review and research proposals. Simulation & Gaming, 40(2), 217–266.

    Article  Google Scholar 

  • Younis, B., & Loh, C. S. (2010). Integrating serious games in higher education programs. Proceedings of the academic colloquium 2010: Building partnership in teaching excellence. Ramallah, Palestine: AMIDEAST.

    Google Scholar 

  • Zoeller, G. (2010). Development telemetry in video games projects. Paper presented at the Game Developer Conference 2010. San Francisco, CA. Retrieved March 1, 2011 from http://gdc.gulbsoft.org/talk.

Download references

Acknowledgments

The research work in this article is made possible in part through funding from the Defence University Research Instrumentation Program (DURIP) grant from the US Army Research Office (ARO). Screenshots of PeTRA (Figs. 8.2 and 8.5) are used with permission from the author.

Author information

Authors and Affiliations

  1. Virtual Environment Lab (V-Lab), Department of Curriculum and Instruction, Southern Illinois University Carbondale, 625 Wham Drive, 4610, Carbondale, IL, 62901, USA

    Christian S. Loh

Authors
  1. Christian S. Loh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian S. Loh .

Editor information

Editors and Affiliations

  1. , Fakultät für Sozialwissenschaften, University of Mannheim, Gebaudeteil B A5,6, Mannheim, 68131, Baden-Württemberg, Germany

    Dirk Ifenthaler

  2. , College of Education, University of Oklahoma, Van Vleet Oval 820, Norman, 73019, Oklahoma, USA

    Deniz Eseryel

  3. Van Fleet Oval 820, Norman, 73019, Oklahoma, USA

    Xun Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media New York

About this chapter

Cite this chapter

Loh, C.S. (2012). Information Trails: In-Process Assessment of Game-Based Learning. In: Ifenthaler, D., Eseryel, D., Ge, X. (eds) Assessment in Game-Based Learning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3546-4_8

Download citation

Publish with us

Policies and ethics

Search

Navigation