Skip to main content
SpringerLink
Log in

Measurement Principles for Gaming

  • Chapter
  • First Online:
Assessment in Game-Based Learning

  • 3846 Accesses

Abstract

This book chapter examines the rapidly emerging field of computer-based assessment for gaming from the perspective of sound measurement principles. In educational gaming, a variety of assessment approaches are being used to make inferences about the learner. Evidence is used in some settings to drive adaptive delivery of content, score interactions and gaming products, produce a variety of reports, and, increasingly, provide information for educational decision-makers that impacts students in a variety of subject matter areas and grade levels. In this chapter, we examine innovative approaches for use of assessment evidence through gaming. The principles of the Berkeley Evaluation and Assessment Research Center Assessment System are used as a framework of analysis for these products with respect to key measurement principles. Some directions of future game development are presented, particularly for problem-solving and other complex teaching and learning settings. Suggestions are provided of promising emerging avenues for game developers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Behrens, J. T., Frezzo, D. C., Mislevy, R. J., Kroopnick, M., & Wise, D. (2008). Structural, functional, and semiotic symmetries in simulation-based games and assessments. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 59–80). New York: Erlbaum.

    Google Scholar 

  • Bennett, R. E. (1990). Toward intelligent assessment: An integration of constructed response testing, artificial intelligence, and model-based measurement. Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Biggs, J. (1999). Teaching for Quality Learning at University. Buckingham: SRHE and Open University Press.

    Google Scholar 

  • Center for Applied Special Technology. (2009). UDL guidelines 1.0. Retrieved June 20, 2009, from http://www.cast.org/publications/index.html.

    Google Scholar 

  • Clark, D., Nelson, B., Sengupta, P., & D’Angelo, C. (2009). Rethinking science learning through digital games and simulations: Genres, examples, and evidence. An NAS commissioned paper. Nov. 11, 2011. http://www7.nationalacademies.org/bose/Clark_Gaming_CommissionedPaper.pdf

  • de Freitas, S. (2006). Using games and simulations for supporting learning. Learning, Media and Technology, 31(4), 343–358.

    Article  Google Scholar 

  • de Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers in Education, 46, 249–264.

    Article  Google Scholar 

  • Drachen, A., Canossa, A., & Yannakakis, G. N. (2009). Player modeling using self-organization in tomb raider: Underworld. In Paper presented at the CIG—IEEE symposium on computational intelligence and games, Milano, Italy.

    Google Scholar 

  • Epistemic Games. (2011). What are epistemic games? (1). Retrieved from May 29, 2011 http://epistemicgames.org/eg/?cat=5.

  • Frezzo, D. C., Behrens, J. T., & Mislevy, R. M. (2010). Design patterns for learning and assessment: Facilitating the introduction of a complex simulation-based learning environment into a community of instructors. Journal of Science Education and Technology, 19, 105–114.

    Article  Google Scholar 

  • Galas, C., & Ketelhut, D. J. (2006). River City, the MUVE. Learning and Leading with Technology, 33(7), 31–32.

    Google Scholar 

  • Gee, J. P. (2003). High score education. Wired Magazine, 11(5), 1–2.

    Google Scholar 

  • Gee, J. P., & Shaffer, D. W. (2010). Looking where the light is bad: Video games and the future of assessment, epistemic games group working paper 2010-02. Madison: University of Wisconsin-Madison.

    Google Scholar 

  • Gredler, M. E. (1996). Educational games and simulations: A technology in search of a research paradigm. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 521–539). New York: MacMillan.

    Google Scholar 

  • Hariri, B., Shirmohammadi, S., & Reza Pakravan, M. (2008). A hierarchical HMM model for online gaming traffic patterns. In Paper presented at the IEEE: I²MTC 2008—IEEE international instrumentation and measurement technology conference. Victoria, Vancouver Island, Canada.

    Google Scholar 

  • Harris, A. L., & Marsh, K. (2010). Is a raceless identity an effective strategy for academic success among blacks. Social Science Quarterly, 91, 1242–1263.

    Article  Google Scholar 

  • Hemmings, A. (1998). The self transformations of African-American achievers. Youth and Society, 29(2), 330–368.

    Article  Google Scholar 

  • Hu, W. (2011, January 4). Math That Moves: Schools Embrace the iPad, New York Times. Retrieved from http://www.nytimes.com/2011/01/05/education/05tablets.html?_r=1&pagewanted=1&ref=education).

  • Huffaker, D., Wang, J., Treem, J., Ahmad, M., Fullerton, L., Williams, D., et al. (2009). The social behaviors of experts in massive multiplayer online role-playing games. In Paper presented at the 2009 IEEE Social Computing (SocialCom-09). Symposium on Social Intelligence and Networking (SIN-09), Vancouver, Canada.

    Google Scholar 

  • Jantke, K. P., & Gaudl, S. (2010). Taxonomic contributions to digital games science. In Paper presented at the games innovations conference (ICE-GIC). Hong Kong

    Google Scholar 

  • Kafai, Y. B. (2006). Playing and making games for learning: Instructionist and constructionist perspectives for game studies. Games and Culture, 1(1), 34–40.

    Article  Google Scholar 

  • Ketelhut, D. J., Dede, C., Clarke, J., Nelson, B., & Bowman, C. (2007). Studying situated learning in a multi-user virtual environment. In E. Baker, J. Dickieson, W. Wulfeck, & H. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 37–58). Mahwah, NJ: Erlbaum Associates.

    Google Scholar 

  • Kirriemuir, J. (2002). Video gaming, education and digital learning technologies: Relevance and opportunities. D-Lib Magazine, 8(2), 1–14.

    Google Scholar 

  • Klabbers, J. H. G. (2006). The magic circle: Principles of gaming & simulation. Rotterdam: Sense Publishers.

    Google Scholar 

  • Lieberman, D. A. (2006). What can we learn from playing interactive games? In P. Vorderer & J. Bryant (Eds.), Playing video games: Motives, responses, and consequences (pp. 379–397). Mahwah, NJ: Erlbaum Associates.

    Google Scholar 

  • Lieberman, D. A., Fisk, M. C., & Biely, E. (2009). Digital games for young children ages three to six: From research to design. Computers in the Schools, 26(4), 299–313.

    Article  Google Scholar 

  • Linn, R. L. & Baker, E. L. (1996). “Can performance-based student assessments be psychometrically sound?” In J. B. Baron & D. P. Wolf (Eds.), Performance-based student assessment: Challenges and possibilities. Ninety-fifth Yearbook of the National Society for the Study of Education, Part In (pp. 84–103). Chicago: University of Chicago Press.

    Google Scholar 

  • Linn, R.L. & Baker, E.L. (1996). “Can performance-based student assessments be psychometrically sound?” In J.B. Baron & D.P. Wolf (Eds.), Performance-based student assessment: Challenges and possibilities. Ninety-fifth Yearbook of the National Society for the Study of Education, Part In (pp. 84-103). Chicago: University of Chicago Press.

    Article  Google Scholar 

  • Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. Los Angeles, CA: CRESST.

    Google Scholar 

  • National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

    Google Scholar 

  • Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In B. R. Gifford & M. C. O’Connor (Eds.), Changing assessments: Alternative views of aptitude, achievement and instruction (pp. 37–76). Boston, MA: Kluwer Academic.

    Google Scholar 

  • Scalise, K. (2011). Creating innovative assessment items and test forms including through simulations and gaming. In R. W. Lissitz & H. Jiao (Eds.), Computers and Their Impact on State Assessment: Recent History and Predictions for the Future. Charlotte: Information Age Publishing.

    Google Scholar 

  • Scalise, K., Bernbaum, D. J., Timms, M. J., Veeragoudar Harrell, S., Burmester, K., Kennedy, C. A., et al. (2007). Adaptive technology for e-learning: Principles and case studies of an emerging field. Journal of the American Society for Information Science and Technology, 58(14), 1–15.

    Article  Google Scholar 

  • Scalise, K., Timms, M., Clark, L., & Moorjani, A. (2009). Student learning in science simulations: What makes a difference. In Paper presented at the conversation, argumentation, and engagement and science learning, American Educational Research Association, San Diego, CA.

    Google Scholar 

  • Shaffer, D. W., & Graesser, A. (2010). Using a quantitative model of participation in a community of practice to direct automated mentoring in an ill-formed domain. In Paper presented at the Intelligent Tutoring Systems (ITS). May 29, 2011. Retrieved from http://epistemicgames.org/eg/wp-content/uploads/ITS-workshop-shaffer-graesser-041510.pdf

  • Shute, V., Masduki, I., & Donmez, O. (2010). Conceptual framework for modeling, assessing, and supporting competencies within game environments. Technology, Instruction, Cognition and Learning, 8(2), 137–161.

    Google Scholar 

  • Shute, V., Maskduki, I., Donmez, O., Kim, Y. J., Dennen, V. P., Jeong, A. C., et al. (2010). Chapter 15. Modeling, assessing, and supporting key competencies within game environments. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 281–309). New York, NY: Springer.

    Chapter  Google Scholar 

  • Shute, V., Rieber, L., & Van Eck, R. (2010). Games…and…learning. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (3rd ed., pp. 321–332). Boston, MA: Pearson Education.

    Google Scholar 

  • Shute, V., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Chapter 18. Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 297–321). London: Routledge.

    Google Scholar 

  • Squire, K. D. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19–29.

    Article  Google Scholar 

  • Steinkuehler, C. (2008). Massively multiplayer online games as an educational technology: An outline for research. Educational Technology, 48(1), 10–21.

    Google Scholar 

  • Stenhouse, D. (1986). Conceptual change in science education: Paradigms and language-games. Science Education, 70(4), 413–425.

    Article  Google Scholar 

  • Thai, A., Lowenstein, D., Ching, D., & Rejeski, D. (2009). Game changer: Investing in digital play to advance children’s learning and health. New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.

    Google Scholar 

  • The NPD Group. (2010). 2010 Total consumer spend on all games content in the U.S. estimated between $15.4 to $15.6 bBillion. Inc, Port Washington, NY.

    Google Scholar 

  • Vogel, J. F., Vogel, D. S., Cannon-Bowers, J. A., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229–243.

    Article  Google Scholar 

  • Weisburgh, M. (2004). Documenting good education and training practices through design patterns, international forum of educational technology & society. Retrieved May 28, 2011, from http://ifets.ieee.org/discussions/discuss_june2004.html

  • Williamson, B. (2009). Computer games, schools, and young people. Bristol: FutureLab Series.

    Google Scholar 

  • Wilson, M., & Adams, R. J. (1995). Rasch Models for Item Bundles. Psychometrika, 60(2), 181–198.

    Google Scholar 

  • Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Wilson, M., Bejar, I., Scalise, K., Templin, J., Wiliam, D., & Torres Irribarra, D. (2010a). 21st-century measurement for 21st-century skills. In Paper presented at the American educational research association annual meeting, Denver, CO.

    Google Scholar 

  • Wilson, M., Bejar, I., Scalise, K., Templin, J., Wiliam, D., & Torres Irribarra, D. (2010b). Assessment and teaching of 21st century skills: Perspectives on methodological issues. In White paper presented at the Learning and Technology World Forum 2010, London.

    Google Scholar 

  • Wilson, M., & Sloane, K. (2000). From principles to practice: An embedded assessment system. Applied Measurement in Education, 13(2), 181–208.

    Article  Google Scholar 

  • Young, M. F., Schrader, P. G., & Zheng, D. (2006). MMOGs as learning environments: An ecological journey into Quest Atlantis and Sims Online. Innovate: Journal of Online Education, 2(4). Retrieved April 1, 2011, from http://www.innovateonline.info/index.php?view=article%26id=66

Download references

Author information

Authors and Affiliations

  1. Educational Methodology, Policy and Leadership, University of Oregon, 4716 Bennett Valley Road, Santa Rosa, CA, 95404, USA

    Kathleen Scalise

  2. University of California, Berkeley, USA

    Mark Wilson (Mark Wilson, Professor)

Authors
  1. Kathleen Scalise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathleen Scalise .

Editor information

Editors and Affiliations

  1. , Fakultät für Sozialwissenschaften, University of Mannheim, Gebaudeteil B A5,6, Mannheim, 68131, Baden-Württemberg, Germany

    Dirk Ifenthaler

  2. , College of Education, University of Oklahoma, Van Vleet Oval 820, Norman, 73019, Oklahoma, USA

    Deniz Eseryel

  3. Van Fleet Oval 820, Norman, 73019, Oklahoma, USA

    Xun Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media New York

About this chapter

Cite this chapter

Scalise, K., Wilson, M. (2012). Measurement Principles for Gaming. In: Ifenthaler, D., Eseryel, D., Ge, X. (eds) Assessment in Game-Based Learning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3546-4_15

Download citation

Publish with us

Policies and ethics

Search

Navigation