Digital Knowledge Maps: The Foundation for Learning Analytics Through Instructional Games
The CyGaMEs (Cyberlearning through Game-based, Metaphor Enhanced Learning Objects) approach to instructional game design and embedded assessment provides a formalism to translate domain knowledge into procedural gameplay. As such, CyGaMEs learning environments are transactional digital knowledge maps that make abstract concepts concrete and actionable: translating what experts know into procedures learners do (discover and apply). CyGaMEs produces games designed to provide viable prior knowledge as preparation for future learning. After knowledge specification through a task analysis, the method applies cognitive science analogical reasoning theory to translate targeted learning goals into game goals and translate targeted knowledge as the game world (e.g., rules and core mechanics). The CyGaMEs approach designs gameplay parameters as the Timed Report measure of player performance to quantify and trace trajectories of learning and achievement. The approach is one way to address design for alignment and shortcomings and limitations documented in the literature that plague current learning game design, embedded assessment, and research. Chapter discussion introduces the national initiative for cyberlearning and embedded assessment and insights from evidence-centered design and cognitive tutor development practices, especially regarding task analysis and cognitive task analysis. Then CyGaMEs’ Selene: A Lunar Construction GaME design artifacts, screen captures, gameplay data, and analyses illustrate this approach to design and embedded assessment. A case is made that instructional game design with embedded assessment is an enterprise requiring complex expertise among teams of professionals—topped by talent and creativity.
KeywordsTask analysis Timed Report Learning analytics Instructional games CyGaMEs
Grateful appreciation to the CyGaMEs team (Charles A. Wood, Robert E. Kosko, Barbara G. Tabachnick, Douglas Moore, Janis Worklan, Cassie Lightfritz, Nieves Leticia Martín Hernández, Ronald Magers, Matthew Petrole, and Steven Nowak), CyGaMEs Junior Research Advisory Council (JRAC) member Gabrielle Ménard, and all of the CyGaMEs recruiters and players.
This research was supported by National Science Foundation grant DRL-0814512. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Portions of this [Selene] software are provided under license from Second Avenue Software Inc., copyright 2007–2010 Second Avenue Software Inc. All rights reserved.
- American Association for the Advancement of Science. (2001). Atlas of science literacy (Vol. 1). Washington, DC: American Association for the Advancement of Science and National Science Teachers Association.Google Scholar
- American Association for the Advancement of Science. (2007). Atlas of science literacy. Washington, DC: American Association for the Advancement of Science and National Science Teachers Association.Google Scholar
- Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum.Google Scholar
- Anderson, J. R., & Schunn, C. D. (2000). Implications of the ACT-R learning theory: No magic bullets. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 5). Mahwah, NJ: Erlbaum.Google Scholar
- Baker, E. L., Chung, G. K. W. K., & Delacruz, G. C. (2007). Design and validation of technology-based performance assessments. In J. M. Spector, M. D. Merrill, J. J. G. Merriënboer, & M. R. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 595–604). Mahwah, NJ: Erlbaum.Google Scholar
- Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics. Washington, DC: Retrieved from U.S. Department of Education: Office of Educational Technology website: http://www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf.Google Scholar
- Borgman, C. L., Abelson, H., Johnson, R., Koedinger, K. R., Linn, M. C., Lynch, C. A., Szalay, A. (2008). Fostering learning in the networked world: The cyberlearning opportunity and challenge: A 21st century agenda for the National Science Foundation. Arlington, VA. Retrieved June 24, 2012 from National Science Foundation website: http://www.nsf.gov/pubs/2008/nsf08204/nsf08204.pdf?govDel=USNSF_124.
- Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.Google Scholar
- Clark, R. E., Feldon, D. F., Merriënboer, J. J. G., Yates, K. A., & Early, S. (2008). Cognitive task analysis. In J. M. Spector, M. D. Merrill, J. van Merriënboer, & M. R. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577–593). New York: Lawrence Erlbaum.Google Scholar
- Committee on Conceptual Framework for the New K-12 Science Education Standards, & National Research Council. (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC. Retrieved from National Academies Press website: http://www.nap.edu/catalog.php?record_id=13165.
- E-Line Media, & Institute of Play. (2013). Gamestar mechanic. Retrieved January 4, 2013 from http://gamestarmechanic.com/.
- Fullerton, T. (2008). Game design workshop: A Playcentric approach to creating innovative games (2nd ed.). Burlington, MA: Elsevier.Google Scholar
- Fullerton, T., Swain, C., & Hoffman, S. (2004). Game design workshop: Designing, prototyping, and playtesting games. San Francisco, CA: CMP Books.Google Scholar
- Gagné, R. M. (1965). The conditions of learning. New York: Holt, Rinehart, & Winston.Google Scholar
- Gagné, R. M. (2012/2000/1968). Learning hierarchies. In R. C. Richey (Ed.), The legacy of Robert M. Gagné (pp. 63-84). Tulsa, OK: The ERIC Clearinghouse on Information & Technology and International Board of Standards for Training, Performance, and Instruction. Retrieved from http://www.ibstpi.org/Products/pdf/chapter_2.pdf. (Reprinted from: Learning Hierarchies. Experimental Psychologist, 6, 1–9, by Robert M. Gagné, 1968.).
- Gagné, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of instructional design (4th ed.). Belmont, CA: Wadsworth/Thomson Learning.Google Scholar
- Gentner, D. (1980). The structure of analogical models in science (report no. 4451, NTIS no. AD-A087-625). Springfield, VA: National Technical Information Service, U.S. Department of Commerce.Google Scholar
- Holyoak, K. J., Gentner, D., & Kokinov, B. N. (2001). Introduction: The place of analogy in cognition. In D. Gentner, K. J. Holyoak, & B. N. Kokinov (Eds.), The analogical mind: Perspectives from cognitive science (pp. 1–20). Cambridge, MA: MIT Press.Google Scholar
- Klopfer, E., Osterweil, S., & Salen, K. (2009). Moving learning games forward: Obstacles, opportunities, and openness. Boston: The Education Arcade, Massachusetts Institute of Technology.Google Scholar
- Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: The University of Chicago Press.Google Scholar
- Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books.Google Scholar
- Mislevy, R. J. (2011). Evidence-Centered Design for simulation-based assessment. (CRESST Report 800). Los Angeles. Retrieved from University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST) website: http://www.cse.ucla.edu/products/reports/R800.pdf.
- Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. (CSE 597). Los Angeles. Retrieved from University of California Center for the Study of Evaluation, National Center for Research on Evaluation website: http://www.cse.ucla.edu/products/reports/TR597.pdf.
- National Research Council. (2011). Learning science through computer games and simulations. In Committee on Science Learning: Computer Games, Simulations, and Education. M. A. Honey and M. L. Hilton (Eds.). Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press.Google Scholar
- National Research Council Committee on the Foundations of Assessment, Board on Testing and Assessment, C. f. E., & Division of Behavioral and Social Sciences and Education. (2001). Knowing what students know: The science and design of educational assessment. In J. Pelligrino, N. Chudowsky, & R. Glaser (Eds.). Washington, DC: National Academy Press. Retrieved from http://www.nap.edu/books/0309072727/html/.
- National Science Foundation. (2012a). Building community and capacity for data-intensive research in the social, behavioral, and economic sciences and in education and human resources (BBC-SBE/EHR): program solicitation NSF 12-538. Arlington, VA: Author.Google Scholar
- National Science Foundation. (2012b, June 19, 2012). Cyberinfrastructure framework for 21st century science and engineering (CIF21). Retrieved July 7, 2012 from http://www.nsf.gov/funding/pgmsumm.jsp?pims_id=504730.
- Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.Google Scholar
- Next Generation Science Standards Team. (2012). Next generation science standards (Draft). Washington, DC: Achieve Inc.Google Scholar
- Polya, G. (1954). Mathematics and plausible reasoning: Volume 1: Induction and analogy in mathematics (Vol. 1). Princeton, NJ: Princeton University Press.Google Scholar
- Quellmalz, E. S., Timms, M. J., & Schneider, S. A. (2009). Assessment of student learning in science simulations and games. Paper commissioned for the National Research Council Workshop on gaming and simulations, October 6–7. Washington, DC. Retrieved from http://www7.nationalacademies.org/bose/Schneider_Gaming_CommissionedPaper.pdf.
- Reese, D. D. (2003a). Metaphor and content: An embodied paradigm for learning. Ph.D. Dissertation, Virginia Polytechnic Institute and State University, Blacksburg, VA. Retrieved from http://scholar.lib.vt.edu/theses/available/etd-03312003-115151/unrestricted/Reese_D_D.pdf.
- Reese, D. D. (2003b). Trees of knowledge: Changing mental models through metaphorical episodes and concept maps. In R. E. Griffin, V. S. Williams, & J. Lee (Eds.), Turning trees: Selected readings (pp. 205–214). Tempe, AZ: International Visual Literacy Association.Google Scholar
- Reese, D. D. (2009). Structure mapping theory as a formalism for instructional game design and assessment. In B. Kokinov, K. Holyoak, & D. Gentner (Eds.), New frontiers in analogy research: Proceedings of the 2nd International Conference on Analogy (Analogy '09) (pp. 394–403). Sofia, Bulgaria: New Bulgarian University Press.Google Scholar
- Reese, D. D., & Coffield, J. (2005). Just-in-time conceptual scaffolding: Engineering sound instructional metaphors. International Journal of Technology, Knowledge, and Society, 1(4), 183–198.Google Scholar
- Reese, D. D., Seward, R. J., Tabachnick, B. G., Hitt, B., Harrison, A., & McFarland, L. (2012). Timed Report measures learning: Game-based embedded assessment. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 145–172). New York: Springer.CrossRefGoogle Scholar
- Reese, D. D., & Tabachnick, B. G. (2010). The moment of learning: Quantitative analysis of exemplar gameplay supports CyGaMEs approach to embedded assessment. In J. Earle (Ed.), Building a knowledge base to inform educational practice in STEM: Examples from the REESE portfolio. Symposium conducted at the annual meeting of the Society for Research on Educational Effectiveness 2010, Washington, DC. Structured abstract retrieved from http://www.sree.org/conferences/2010/program/abstracts/191.pdf.
- Richland, L. E., Stigler, J. W., & Holyoak, K. J. (2012). Teaching conceptual structure of mathematics. Educational Psychologist, 47(3). doi: 10.1080/00461520.2012.667065.
- Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. The Journal of Technology, Learning, and Assessment, 8(4). Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/download/1623/1467.
- Salen, K. (2007). Gaming literacy studies: A game design study in action. Journal of Educational Media and Hypermedia, 16(3), 301–322.Google Scholar
- Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press.Google Scholar
- Salthouse, T. A. (1991). Expertise as the circumvention of human processing limitations. In K. A. Ericcson & J. Smith (Eds.), Toward a general theory of expertise (pp. 286–300). Cambridge, England: Cambridge University Press.Google Scholar
- Schell, J. (2008). The art of game design: A book of lenses. New York: Elsevier.Google Scholar
- Shulman, L. E. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–22.Google Scholar
- Smith, P. L., & Ragan, T. J. (1993). Instructional design (1st ed.). New York: Merrill.Google Scholar
- Smith, P. L., & Ragan, T. J. (2005). Instructional design (3rd ed.). Hoboken, NJ: John Wiley & Sons.Google Scholar
- Timms, M., Clements, D. H., Gobert, J., Ketelhut, D. J., Lester, J. C., Reese, D. D., & Wiebe, E. (2012). New measurement paradigms. Retrieved from http://cadrek12.org/sites/default/files/NMPReport0414120.pdf.
- U.S. Department of Education Office of Educational Technology. (2010). Transforming American education: Learning powered by technology: National education technology plan 2010. Washington, DC: Author.Google Scholar
- Wood, C. A. (2003). The modern Moon: A personal view. Cambridge, MA: Sky Publishing Corporation.Google Scholar
- Wood, C. A., Lorenz, R., Kirk, R., Lopes, R., Mitchell, K., Stofan, E., & Cassini RADAR Team. (2010). Impact craters on Titan. Icarus, 206(1), 540–558. doi: 10.1016/j.icarus.2009.08.021
- ZeptoLab. (2012). Cut the Rope developer candy: Behind the scenes. Retrieved October 12, 2012 from http://www.youtube.com/watch?v=pxn1pNzEwI.