Skip to main content
SpringerLink
Log in

Alternative Assessment Strategies for Complex Problem Solving in Game-Based Learning Environments

  • Chapter
  • First Online:
Multiple Perspectives on Problem Solving and Learning in the Digital Age

Abstract

The central thesis of this chapter is that emerging technologies such as digital games compel educators, educational researchers, and instructional designers to conceptualize learning, instruction, and assessment in fundamentally different ways. New technologies, including massively multi-player digital games offer new opportunities for learning and instruction; however, there is as yet insufficient evidence to support sustained impact on learning and instruction, apart from the case of military training based on large simulated war games. Technologically sophisticated design and assessment frameworks are likely to facilitate progress in this area, and that is our focus in this chapter. Specifically, we provide an integrated framework for assessing complex problem solving in digital game-based learning in the context of a longitudinal design-based research study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2002). Enhancing the design and delivery of assessment systems: A four process architecture. Journal of Technology, Learning, and Assessment, 1(5), 3–63.

    Google Scholar 

  • Baker, E. L., Chung, G. K. W. K., & Delacruz, G. C. (2008). Design and validation of technology-based performance assessments. In J. M. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 595–604). New York: Taylor & Francis Group.

    Google Scholar 

  • Barab, S. A., & Duffy, T. (2000). From practice fields to communities of practice. In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. 25–56). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Barab, S. A., Tomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86–107.

    Google Scholar 

  • Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

    Google Scholar 

  • Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315.

    Google Scholar 

  • Chi, M. T. H., & Glaser, R. (1985). Problem solving ability. In R. J. Sternberg (Ed.), Human abilities: An information processing approach (pp. 227–257). San Francisco: W. H. Freeman & Co.

    Google Scholar 

  • Chin, J., Dukes, R., & Gamson, W. (2009). Assessment in simulation and gaming: A review of the last 40 years. Simulation & Gaming,40, 553–568.

    Google Scholar 

  • Chung, G. K. W. K., & Baker, E. L. (2003). An exploratory study to examine the feasibility of measuring problem-solving processes using a click-through interface. Journal of Technology, Learning and Assessment, 2(2). Retrieved April 22, 2010, from http://www.jtla.org.

  • Dickey, M. D. (2005). Engaging by design: How engagement strategies in popular computer and video games can inform instructional design. Educational Technology Research and Development, 53(2), 67–83.

    Google Scholar 

  • Dummer, P., & Ifenthaler, D. (2005). Planning and assessing navigation in model-centered learning environments. Why learners often do not follow the path laid out for them. In G. Chiazzese, M. Allegra, A. Chifari, & S. Ottaviano (Eds.), Methods and technologies for learning (pp. 327–334). Sothhampton: WIT Press.

    Google Scholar 

  • Dunkle, M. E., Schraw, G., & Bendixen, L. D. (1995). Cognitive processes in well-structured and ill-defined problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.

    Google Scholar 

  • Eigler, G. (2005). Textproduzieren als Wissensnutzungs- und Wissenserwerbsstrategie. In H. Mandl & H. F. Friedrich (Eds.), Handbuch Lernstrategien (pp. 187–205). Göttingen: Hogrefe.

    Google Scholar 

  • Eliaa, I., Gagatsisa, A., & Demetriou, A. (2007). The effects of different modes of representation on the solution of one-step additive problems. Learning and Instruction, 17(6), 658–672.

    Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review,87, 215–251.

    Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

    Google Scholar 

  • Eseryel, D. (2006). Expert conceptualizations of the domain of instructional design: An investigative study on the DEEP assessment methodology for complex problem-solving outcomes. Unpublished doctoral dissertation, Syracuse University.

    Google Scholar 

  • Eseryel, D., Ge, X., Law, V., Hayes, T., Guo, Y., & Ifenthaler, D. (2009). The effects of an educational massively multiplayer online game on students’ complex problem solving skill acquisition. Paper presented at the Annual Convention of Association for Educational Communications and Technology, Louisville, KY.

    Google Scholar 

  • Funke, J. (1985). Steuerung dynamischer Prozesse durch Aufbau und Anwendung subjektiver Kausalmodelle. Zeitschrift für Psychologie, 193(4), 443–465.

    Google Scholar 

  • Galarneau, L., & Zibit, M. (2007). Online games for 21st century skills. In D. Gibson, C. Aldrich, & M. Prensky (Eds.), Games and simulations in online learning: Research and development frameworks (pp. 59–88). Hershey, PA: Information Science Publishing.

    Google Scholar 

  • Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave-Macmillan.

    Google Scholar 

  • Gick, M. L. (1986). Problem-solving strategies. Educational Psychologist, 21(1&2), 99–120.

    Google Scholar 

  • Hardy, I., & Stadelhofer, B. (2006). Concept Maps wirkungsvoll als Strukturierungshilfen einsetzen. Welche Rolle spielt die Selbstkonstruktion? Zeitschrift für Pädagogische Psychologie, 20(3), 175–187.

    Google Scholar 

  • Herl, H. E., Baker, E. L., & Niemi, D. (1996). Construct validation of an approach to modeling cognitive structure of U.S. history knowledge. Journal of Educational Research, 89(4), 206–218.

    Google Scholar 

  • Hong, N. S., Jonassen, D. H., & McGee, S. (2003). Predictors of well-structured and ill-structured problem solving in an astronomy simulation. Journal of Research in Science Teaching, 40(1), 6–33.

    Google Scholar 

  • Ifenthaler, D. (2008). Practical solutions for the diagnosis of progressing mental models. In D. Ifenthaler, P. Pirnay-Dummer, & J. M. Spector (Eds.), Understanding models for learning and instruction. Essays in honor of Norbert M. Seel (pp. 43–61). New York: Springer.

    Google Scholar 

  • Ifenthaler, D. (2009a). Bridging the gap between expert-novice differences: The model-based feedback approach. In Kinshuk, D. G. Sampson, J. M. Spector, P. Isaias, & D. Ifenthaler (Eds.), Proceedings of the IADIS international conference on cognition and exploratory learning in the digital age (pp. 49–60), Rome: IADIS Press.

    Google Scholar 

  • Ifenthaler, D. (2009b). Model-based feedback for improving expertise and expert performance. Technology, Instruction, Cognition and Learning, 7(2), 83–101.

    Google Scholar 

  • Ifenthaler, D. (2009c). Using a causal model for the design and development of a simulation game for teacher education. Technology, Instruction, Cognition and Learning, 6(3), 193–212.

    Google Scholar 

  • Ifenthaler, D. (2010a). Learning and instruction in the digital age. In J. M. Spector, D. Ifenthaler, P. Isaías, Kinshuk, & D. G. Sampson (Eds.), Learning and instruction in the digital age: Making a difference through cognitive approaches, technology-facilitated collaboration and assessment, and personalized communications (pp. 3–10). New York: Springer.

    Google Scholar 

  • Ifenthaler, D. (2010b). Relational, structural, and semantic analysis of graphical representations and concept maps. Educational Technology Research and Development, 58(1), 81–97.

    Google Scholar 

  • Ifenthaler, D. (2010c). Scope of graphical indices in educational diagnostics. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 213–234). New York: Springer.

    Google Scholar 

  • Ifenthaler, D., Masduki, I., & Seel, N. M. (2009). The mystery of cognitive structure and how we can detect it. Tracking the development of cognitive structures over time. Instructional Science. Retrieved October 18, 2010, from http://www.springerlink.com/content/y62024472v394468/

  • Ifenthaler, D., & Pirnay-Dummer, P. (2009). Assessment of knowledge: Do graphical notes and texts represent different things? In M. R. Simonson (Ed.), Annual proceedings of selected research and development papers presented at the national convention of the Association for Educational Communications and Technology (32nd, Louisville, KY, 2009), (Vol. 2, pp. 86–93). Bloomington, IN: AECT.

    Google Scholar 

  • Ifenthaler, D., Pirnay-Dummer, P., & Seel, N. M. (Eds.). (2010). Computer-based diagnostics and systematic analysis of knowledge. New York: Springer.

    Google Scholar 

  • Ifenthaler, D., & Seel, N. M. (2005). The measurement of change: Learning-dependent progression of mental models. Technology, Instruction, Cognition and Learning, 2(4), 317–336.

    Google Scholar 

  • Johnson-Laird, P. N. (1989). Mental models. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 469–499). Cambridge, MA: MIT Press.

    Google Scholar 

  • Johnson, T. E., Ifenthaler, D., Pirnay-Dummer, P., & Spector, J. M. (2009). Using concept maps to assess individuals and team in collaborative learning environments. In P. L. Torres & R. C. V. Marriott (Eds.), Handbook of research on collaborative learning using concept mapping (pp. 358–381). Hershey, PA: Information Science Publishing.

    Google Scholar 

  • Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.

    Google Scholar 

  • Klabbers, J. H. (Ed.). (2008). The magic circle: Principles of gaming & simulation. Rotterdam: Sense.

    Google Scholar 

  • Koschmann, T. (Ed.). (1996). CSCL, theory and practice of an emerging paradigm. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Macy, M., Squires, J. K., & Barton, E. E. (2009). Providing optimal opportunities: Structuring practicum experiences in early intervention and early childhood special education preservice programs. Topics in Early Childhoos Special Education, 28(4), 209–218.

    Google Scholar 

  • Mandl, H., & Fischer, F. (2000). Mapping-Techniken und Begriffsnetze in Lern- und Kooperationsprozessen. In H. Mandl & F. Fischer (Eds.), Wissen sichtbar machen – Wissensmanagement mit Mapping-Techniken (pp. 3–12). Göttingen: Hogrefe.

    Google Scholar 

  • McFarlane, A. (2003). Editorial. assessment for the digital age, Assessment in Education: Principles, Policy & Practice,10, 261–266.

    Google Scholar 

  • Michael, D., & Chen, S. (2006). Serious games: Games that educate, train, and inform. Boston: Thomson Course Technology PTR.

    Google Scholar 

  • Miller, R. B., Eseryel, D., & Ge, X. (2009). Surviving in space: The effects of a massively multiplayer online game (MMOG) on students’ motivation. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA, USA.

    Google Scholar 

  • Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.

    Google Scholar 

  • Papert, S., & Caperton, G. (1999). Vision for education: The Caperton-Papert platform. Paper presented at the 91st Annual National Governor’ Association Meeting. Retrieved April 22, 2010, from http://www.papert.org/articles/Vision_for_education.html

  • Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2003). Knowing what students know. The secience and design of educational assessment. Washington, DC: National Academy Press.

    Google Scholar 

  • Pirnay-Dummer, P., & Ifenthaler, D. (2010). Automated knowledge visualization and assessment. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 77–115). New York: Springer.

    Google Scholar 

  • Pirnay-Dummer, P., Ifenthaler, D., & Rohde, J. (2009). Text-guided automated self-assessment. In Kinshuk, D. G. Sampson, J. M. Spector, P. Isaias, & D. Ifenthaler (Eds.), Proceedings of the IADIS international conference on cognition and exploratory learning in the digital age (pp. 311–316). Rome: IADIS Press.

    Google Scholar 

  • Pirnay-Dummer, P., Ifenthaler, D., & Spector, J. M. (2010). Highly integrated model assessment technology and tools. Educational Technology Research and Development, 58(1), 3–18.

    Google Scholar 

  • Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.

    Google Scholar 

  • Prensky, M. (2006). Don’t bother me mom–I’m learning. St. Paul, MN: Paragon House.

    Google Scholar 

  • Ruiz-Primo, M. A., Schultz, S. E., Li, M., & Shavelson, R. J. (2001). Comparison of the reliability and validity of scores from two concept-mapping techniques. Journal of Research in Science Teaching, 38(2), 260–278.

    Google Scholar 

  • Scandura, J. M. (1988). Role of relativistic knowledge in intelligent tutoring. Computers in Human Behavior, 4(1), 53–64.

    Google Scholar 

  • Scheele, B., & Groeben, N. (1984). Die Heidelberger Struktur-Lege-Technik (SLT). Eine Dialog-Konsens-Methode zur Erhebung subjektiver Theorien mittlerer Reichweite. Weinheim: Beltz.

    Google Scholar 

  • Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.

    Google Scholar 

  • Seel, N. M. (1999). Educational diagnosis of mental models: Assessment problems and technology-based solutions. Journal of Structural Learning and Intelligent Systems, 14(2), 153–185.

    Google Scholar 

  • Seel, N. M., Ifenthaler, D., & Pirnay-Dummer, P. (2009). Mental models and problem solving: Technological solutions for measurement and assessment of the development of expertise. In P. Blumschein, W. Hung, D. H. Jonassen, & J. Strobel (Eds.), Model-based approaches to learning: Using systems models and simulations to improve understanding and problem solving in complex domains (pp. 17–40). Rotterdam: Sense.

    Google Scholar 

  • Shaffer, D. W. (2006). How computer games help children learn? New York: Palgrave Macmillan.

    Google Scholar 

  • Shute, V. J., & Spector, J. M. (2010). Stealth assessment in virtual worlds. Retrieved April 22, 2010, from http://www.adlnet.gov/Technologies/Evaluation/Library/Additional%20Resources/LETSI%20White%20Papers/Shute%20-%20Stealth%20Assessment%20in%20Virtual%20Worlds.pdf

  • Spector, J. M., Dennen, V. P., & Koszalka, T. A. (2005). Individual and collaborative construction of causal concept maps: an online technique for learning and assessment. In G. Chiazzese, M. Allegra, A. Chifari, & S. Ottaviano (Eds.), Methods and technologies for learning (pp. 223–227). Southampton: WIT Press.

    Google Scholar 

  • Spector, J. M., & Koszalka, T. A. (2004). The DEEP methodology for assessing learning in complex domains (Final report to the National Science Foundation Evaluative Research and Evaluation Capacity Building). Syracuse, NY: Syracuse University.

    Google Scholar 

  • Squire, K. (2004). Replaying history. Unpublished doctoral dissertation, Indiana University, Bloomington, IN.

    Google Scholar 

  • Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10(2–3), 291–330.

    Google Scholar 

  • Swearingen, D., & Eseryel, D. (2010). The effects of massively multiplayer online games on high school students’ mathematics achievement. Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO, USA.

    Google Scholar 

  • Tarciani, E. M., & Clariana, R. B. (2006). A technique for automatically scoring open-ended concept maps. Educational Technology, Research, & Development, 54(1), 65–82.

    Google Scholar 

  • Thomas, D., & Brown, J. S. (2007). The play of imagination: Extending the literary mind. Games & Culture, 2(2), 149–172.

    Google Scholar 

  • Tuzun, H. (2004). Motivating learners in educational computer games. Unpublished doctoral dissertation. Indiana University, Bloomington, IN.

    Google Scholar 

  • Wagner, W., & Wagner, S. U. (1985). Presenting questions, processing responses, and providing feedback in CAI. Journal of Instructional Development, 8(4), 2–8.

    Google Scholar 

  • Warren, S. J., & Dondlinger, M. J. (2009). Designing games for learning. In R. E. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (pp. 1183–1203). Hershey, PA: IGI Global.

    Google Scholar 

  • Willett, J. B. (1988). Questions and answers in the measurement of change. Review of Research in Education, 15, 345–422.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Education, University of Oklahoma, 820 Van Vleet Oval, Norman, OK, 73019, USA

    Deniz Eseryel

  2. Albert-Ludwigs-University Freiburg, Freiburg, Germany

    Dirk Ifenthaler

  3. University of Oklahoma, Norman, OK, USA

    Xun Ge

Authors
  1. Deniz Eseryel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dirk Ifenthaler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xun Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deniz Eseryel .

Editor information

Editors and Affiliations

  1. Abt. Lernforschung, Universität Freiburg, Rempartstr. 11, Freiburg, 79085, Germany

    Dirk Ifenthaler

  2. Learning & Performance Support Lab., University of Georgia, Aderhold Hall 605, Athens, 30602, Georgia, USA

    J. Michael Spector

  3. Universidade Aberta, Rua de Junqueria 86, Lisboa, 1300-344, Portugal

    Pedro Isaias

  4. Dept. Technology in Education &, Digital Systems, University of Piraeus, Androutsou St. 150, Piraeus, 185 32, Greece

    Demetrios Sampson

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Eseryel, D., Ifenthaler, D., Ge, X. (2011). Alternative Assessment Strategies for Complex Problem Solving in Game-Based Learning Environments. In: Ifenthaler, D., Spector, J., Isaias, P., Sampson, D. (eds) Multiple Perspectives on Problem Solving and Learning in the Digital Age. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7612-3_11

Download citation

Publish with us

Policies and ethics

Search

Navigation