Abstract
A recent emphasis has been placed on embedding automated assessment systems within open-ended virtual 3D environments. Several supporting assessment models have been identified to guide the design of technology in support of both teachers and learners, most notably within game-based learning. Current research in automated assessment systems indicates that completeness, accuracy of performance, and timeliness are useful indicators of student learning within simulations. However, a better understanding is needed to guide designers and developers of these systems when facing decisions about where and how to effectively implement the technological structures that support key functions. This chapter identifies a set of design decisions as the result of building an automated assessment prototype within an open-ended 3D environment. The results are expanded through an implementation in a context beyond the prototype, further informing the refinement for an effective development strategy and testing the validity of the proposed system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aldrich, C. (2005). Simulations and the future of learning: An innovative (and perhaps revolutionary) approach to e-learning. San Francisco, CA: Wiley.
Angelo, T. A., & Cross, P. C. (1993). Classroom assessment techniques (2nd ed.). San Francisco: Jossey-Bass.
Banuls, V. A., & Salmeron, J. L. (2007). A scenario-based assessment model—SBAM. Technological Forecasting and Social Change, 74(1), 750–762.
Barab, S., Gresalfi, M., & Arici, A. (2009). Why educators should care about games. Educational Leadership, 67(1), 76–80.
Belland, B. (2011). How to judge if they (are on track to) achieve: The importance of aligning assessment with learning objectives. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives. New York: Springer.
Carolan, T. F., Bilazarian, P., & Nguyen, L. (2005). Automated individual, team, and multi-team performance assessment to support debriefing distributed simulation based exercises (DDSBE) [online report]. Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA460383&Location=U2%26doc=GetTRDoc.pdf.
Driscoll, M. P. (2000). Psychology of learning for instruction (2nd ed.). Needham Heights, MA: Allyn & Bacon.
Elton, L., & Laurillard, D. (1979). Trends in student learning. Studies in Higher Education, 4, 87–102.
Freeman, J., Salter, W. J., & Hoch, S. (2004). The users and functions of debriefing in distributed, simulation-based team training. Human factors and ergonomics society 48th annual meeting, New Orleans, LA.
Gee, J. P. (2008). The ecology of games: Connecting youth, games, and learning. In K. Salen (Ed.), Learning and games (pp. 21–40). Cambridge, MA: MIT Press.
Gibbons, A. S., & Fairweather, G. B. (1998). Computer-based instruction: Design and development. Englewood Cliffs, NJ: Educational Technology Publications.
Gibbons, A. S., & Sommer, S. (2007). Layered design in an instructional simulation. In B. E. Shelton & D. Wiley (Eds.), The design and use of simulation computer games in education (pp. 85–102). Rotterdam, The Netherlands: Sense Publishers.
Keller, J. M. (1993). Motivation by design. Tallahassee: Florida State University.
Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi-user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56–68.
Kirkley, S., & Kirkley, J. (2005). Creating next generation blended learning environments using mixed reality, video games and simulations. TechTrends, 49(3), 42–53.
Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning and instruction, volume 3: Cognitive and affective process analysis. Englewood Cliffs, NJ: Erlbaum.
Mislevy, R. (2011). Evidence centered design as a framework for understanding games and assessments. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives. New York: Springer.
Mislevy, R., Almond, R. G., & Lukas, J. (2004). A brief introduction to evidence-centered design. CSE Technical Report 632, The National Center for Research on Evaluation, Standards, Student Testing (CRESST) [online report]. Retrieved from http://www.cse.ucla.edu/products/reports/r632.pdf.
Nelson, B., Erlandson, B., & Denham, A. (2010). Global channels of evidence for learning and assessment in complex game environments. British Journal of Educational Technology, 42(1), 88–100.
Neville, D., & Shelton, B. E. (2010). Literary and historical 3D-DGBL: Design guidelines. Simulation & Gaming, 41(4), 607–629.
Scoresby, J., & Shelton, B. E. (2007). Using videogame motivation analysis during game play activity aligned with learning objectives. Chicago, IL: American Educational Research Association (AERA).
Scoresby, J., & Shelton, B. E. (2011). Visual perspectives within educational computer games: Effects on presence and flow within virtual learning environments. Instructional Science, 39(3), 227–254.
Shelton, B. E. (2007). Designing instructional games for activity-goal alignment. In B. E. Shelton & D. Wiley (Eds.), The design and use of simulation computer games in education. Rotterdam, The Netherlands: Sense Publishers.
Shelton, B. E., & Scoresby, J. (2011). Aligning game activity with educational goals: Following a constrained design approach to instructional computer games. Educational Technology Research & Development, 59(1), 113–138. Advance online publication. doi: 10.1007/s11423-010-9175-0.
Shelton, B. E., Scoresby, J., Parlin, M. A., & Olsen, J. (2011). Facing the design and development challenges of implementing embedded automated assessment systems in virtual environments. New Orleans, LA: American Educational Research Association (AERA).
Shelton, B. E., Stowell, T., Scoresby, J., Alvarez, M., Capell, M., & Coats, K. C. (2010). A Frankenstein approach to open source: The construction of a 3D game engine as meaningful educational process. IEEE Transactions on Learning Technologies, 3(2), 85–90. doi:ieeecomputersociety.org/10.1109/TLT.2010.3.
Shelton, B. E., & Wiley, D. (2006). Instructional designers take all the fun out of games: Rethinking elements of engagement for designing instructional games. San Francisco, CA: American Educational Research Association (AERA).
Shute, V. J. (1993). A comparison of learning environments: All that glitters. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 47–75). Hillsdale, NJ: Erlbaum.
Silva, E. (2009). Measuring skills for 21st century learning. Phi Delta Kappan, 90(9), 630–634.
Squire, K. (2003). Changing the game: What happens when video games enter the classroom? Innovate, 1(6) [online report]. Retrieved from http://www.innovateonline.info/index.php?view=article%26id=82
Stowell, T., Scoresby, J., Coats, C., Capell, M., & Shelton, B. E. (2009). Utilizing readily available and open source libraries to create a 3D game engine. International Journal of Gaming and Computer-Mediated Simulations, 1(4), 20–49.
Acknowledgments
Portions of this chapter were presented at the 2011 American Educational Research Association international conference. The authors wish to acknowledge Chad Coats, Jon Scoresby, Andrew Hayes, and Jeffrey Olsen for their contributions to this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media New York
About this chapter
Cite this chapter
Shelton, B.E., Parlin, M.A. (2012). Taking Activity-Goal Alignment into Open-Ended Environments: Assessment and Automation in Game-Based Learning. In: Ifenthaler, D., Eseryel, D., Ge, X. (eds) Assessment in Game-Based Learning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3546-4_7
Download citation
DOI: https://doi.org/10.1007/978-1-4614-3546-4_7
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-3545-7
Online ISBN: 978-1-4614-3546-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)