Skip to main content
SpringerLink
Log in

The Role of Construct Definition in the Creation of Formative Assessments in Game-Based Learning

  • Chapter
  • First Online:
Assessment in Game-Based Learning

Abstract

Recently there has been a push to consider assessment, and specifically formative assessment, in conjunction with game-based learning. Formative assessment is meant to provide feedback to students that can guide their learning process. Central to the production of effective formative assessments are the concepts of validity and reliability. Validity refers to the extent to which assessment scores indicate the amount that a learners has of an underlying construct, defined as an unobservable trait. This chapter focuses on the first stages of the process of formative assessment creation—the specification of constructs underlying learning goals. Constructs are specified by the process of theory specification, construct generation, test item development, and construct refinement. By considering construct validity in assessment development, game developers can collect needed data to optimize games and provide evidence of learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.

    Article  Google Scholar 

  • Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238.

    Google Scholar 

  • Belland, B. R., French, B. F., & Ertmer, P. A. (2009). Validity and problem-based learning research: A review of instruments used to assess intended learning outcomes. The Interdisciplinary Journal of Problem-Based Learning, 3(1), 59–89.

    Google Scholar 

  • Blanton, H., & Jaccard, J. (2006). Arbitrary metrics in psychology. American Psychologist, 61(1), 27–41.

    Article  Google Scholar 

  • Clarke, J., & Dede, C. (2009). Design for scalability: A case study of the river city curriculum. Journal of Science Education and Technology, 18, 353–365.

    Article  Google Scholar 

  • Coderre, S., Mandin, H., Harasym, P. H., & Fick, G. H. (2003). Diagnostic reasoning strategies and diagnostic success. Medical Education, 37, 695–703.

    Article  Google Scholar 

  • Conati, C. (2002). Probabilistic assessment of user’s emotions in educational games. Applied Artificial Intelligence, 16(7&8), 555–575.

    Article  Google Scholar 

  • Cronbach, L. J., & Gleser, G. C. (1959). Interpretation of reliability and validity coefficients: Remarks on a paper by Lord. Journal of Educational Psychology, 50(5), 230–237.

    Article  Google Scholar 

  • Culpepper, S. A., & Davenport, E. C. (2009). Assessing differential prediction of college grades by race/ethnicity with a multilevel model. Journal of Educational Measurement, 46(2), 220–242.

    Article  Google Scholar 

  • Delacruz, G. C. (2010). Games as formative assessment environments: Examining the impact of explanations of scoring and incentives on math learning, game performance, and help seeking. Doctoral dissertation, University of California, Los Angeles. UMI number 3446784.

    Google Scholar 

  • DeVoss, V. (1997). Mega math blaster. Teaching Children Mathematics, 4, 120–121.

    Google Scholar 

  • Fletcher, J. D. (2011). Cost analysis in assessing games for learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 417–434). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  • GagnĂ©, R. M., Briggs, L. J., & Wager, W. W. (1988). Principles of instructional design (3rd ed.). New York: Holt, Rinehart, and Winston.

    Google Scholar 

  • Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation and learning: A research and practice model. Simulation & Gaming, 33, 441–467.

    Article  Google Scholar 

  • Gick, M. L. (1986). Problem solving strategies. Educational Psychologist, 21(1&2), 99–120.

    Google Scholar 

  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

    Article  Google Scholar 

  • Jonassen, D. H., & Hernandez-Serrano, J. (2002). Case-based reasoning and instructional design: Using stories to support problem-solving. Educational Technology Research and Development, 50(2), 65–77.

    Article  Google Scholar 

  • Kagan, D. M. (1988). Teaching as clinical problem solving: A critical examination of the analogy and its implications. Review of Educational Research, 58(4), 482–505.

    Google Scholar 

  • Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319–342.

    Google Scholar 

  • Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research (4th ed.). South Melbourne, Australia: Wadsworth.

    Google Scholar 

  • Koenig, A. D., Lee, J. J., Iseli, M., & Wainess, R. (2010). A conceptual framework for assessing performance in games and simulations. CRESST Report 771. Los Angeles: Center for Research on Evaluation, Standards, and Student Testing.

    Google Scholar 

  • Kolodner, J. L. (1993). Case-based reasoning. San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Krause, U., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19, 158–170.

    Article  Google Scholar 

  • Langer, E. J. (1989). Mindfulness. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Langer, E. J. (1993). A mindful education. Educational Psychologist, 28(1), 43–50.

    Article  Google Scholar 

  • Linn, R. L. (2009). Comments on Atkinson and Geiser: Considerations for college admissions testing. Educational Researcher, 38, 677–679.

    Article  Google Scholar 

  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education.

    Google Scholar 

  • Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.

    Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.

    Article  Google Scholar 

  • Moreno-Ger, P., Burgos, D., & Torrente, J. (2009). Digital games in eLearning environments: Current uses and emerging trends. Simulation & Gaming, 40, 669–687.

    Article  Google Scholar 

  • Nelson, B. C., Erlandson, B., & Denham, A. (2011). Global channels of evidence for learning and assessment in complex game environments. British Journal of Educational Technology, 42(1), 88–100.

    Article  Google Scholar 

  • Nunnally, I. H., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill.

    Google Scholar 

  • Quintana, C., Zhang, M., & Krajcik, J. (2005). A framework for supporting metacognitive aspects of online inquiry through software scaffolding. Educational Psychologist, 40(4), 235–244.

    Article  Google Scholar 

  • Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of games for educational purposes: A review of recent research. Simulation & Gaming, 23(3), 261–276.

    Article  Google Scholar 

  • Reynolds, C. R. (2010). Measurement and assessment: An editorial view. Psychological Assessment, 22(1), 1–4.

    Article  Google Scholar 

  • Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. Journal of Technology, Learning, and Assessment, 8(4), 4–47.

    Google Scholar 

  • Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist, 46(1), 26–47.

    Article  Google Scholar 

  • Shelton, B. E., Scoresby, J., Stowell, T., Capell, M. R., Alverez, M. A., & Coats, K. C. (2010). A Frankenstein approach to open source: The construction of a 3D game engine as meaningful educational process. IEEE Transactions on Learning Technologies, 3(2), 85–90.

    Article  Google Scholar 

  • Shepard, L. A. (2009). Commentary: Evaluating the validity of formative and interim assessment. Educational Measurement: Issues and Practice, 28(3), 32–37.

    Article  Google Scholar 

  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78, 153–189.

    Article  Google Scholar 

  • Shute, V., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfield, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295–321). New York: Routledge.

    Google Scholar 

  • Skinner, B. F. (1966). The phylogeny and ontogeny of behavior. Science, 153(3741), 1205–1213.

    Article  Google Scholar 

  • Smith, P., & Ragan, T. (1999). Instructional design. Hoboken, NJ: Wiley.

    Google Scholar 

  • Stone, G. E., Koskey, K. L. K., & Sondergeld, T. A. (2011). Comparing construct definition in the Angoff and Objective Standard Setting models: Playing in a house of cards without a full deck. Educational and Psychological Measurement, 71(6), 942–962.

    Google Scholar 

  • Strauss, M. E., & Smith, G. T. (2009). Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 5, 1–25.

    Article  Google Scholar 

  • Torrente, J., Moreno-Ger, P., Fernandez-Manjon, B., & Sierra, J. (2008). Instructor-oriented authoring tools for educational videogames. In Proceedings of the eighth IEEE international conference on advanced learning technologies (pp. 516–518), Santander, Spain.

    Google Scholar 

  • Vygotsky, L. S. (1962). Thought and language (E. Hanfmann & G. Vakar, Trans.). Cambridge, MA: MIT Press.

    Google Scholar 

  • Walker, A., & Shelton, B. E. (2008). Problem-based educational games: Connections, prescriptions, and assessment. Journal of Interactive Learning Research, 19(4), 663–684.

    Google Scholar 

  • Weisburg, R. W. (1993). Creativity: Beyond the myth of genius. New York: W. H. Freeman.

    Google Scholar 

  • Wiliam, D. (2010a). What counts as evidence in educational achievement? The role of constructs in the pursuit of equity in assessment. Review of Research in Education, 34, 254–284.

    Article  Google Scholar 

  • Wiliam, D. (2010b). Standardized testing and school accountability. Educational Psychologist, 45(2), 107–122.

    Article  Google Scholar 

  • Wiliam, D., & Black, P. (1996). Meanings and consequences: A basis for distinguishing formative and summative functions of assessment? British Educational Research Journal, 22(5), 537–548.

    Article  Google Scholar 

  • Zientek, L. R., Capraro, M. M., & Capraro, R. M. (2008). Reporting practices in quantitative teacher education research: One look at the evidence cited in the AERA panel report. Educational Researcher, 37, 208–216.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Instructional Technology and Learning Sciences, Utah State University, 2830 Old Main Hill, Logan, UT, 84322, USA

    Brian R. Belland

Authors
  1. Brian R. Belland
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian R. Belland .

Editor information

Editors and Affiliations

  1. , Fakultät für Sozialwissenschaften, University of Mannheim, Gebaudeteil B A5,6, Mannheim, 68131, Baden-Württemberg, Germany

    Dirk Ifenthaler

  2. , College of Education, University of Oklahoma, Van Vleet Oval 820, Norman, 73019, Oklahoma, USA

    Deniz Eseryel

  3. Van Fleet Oval 820, Norman, 73019, Oklahoma, USA

    Xun Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media New York

About this chapter

Cite this chapter

Belland, B.R. (2012). The Role of Construct Definition in the Creation of Formative Assessments in Game-Based Learning. In: Ifenthaler, D., Eseryel, D., Ge, X. (eds) Assessment in Game-Based Learning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3546-4_3

Download citation

Publish with us

Policies and ethics

Search

Navigation