Abstract
This effort considers briefly the interpretations and elements of twenty-first century skills, the components of cognitive readiness, and how they may generate overlapping conceptual models for learning and assessment. The chapter then addresses different functions, benefits, and flaws of the different conceptualizations as they apply to various purposes, including the design of learning and assessment. Next, content is treated through the use of a method to develop an ontology that reflects the domain relevant to measurement or learning. Ontologies have at least two major uses. First they provide a graphical representation of verbal goals or domain boundaries where they engender greater transparency, opportunity for agreement, discussion, and support, and sampling plans for critical or highly important content topics and methods. Their more complete use, however, is to set rules and content for the design of databases intended to monitor, revise, and manage performance generated by twenty-first century skill tasks, content, and items given to target respondents. Only a relatively brief treatment is provided here of the development of such smart databases. Finally, the chapter closes with a brief treatment of validity, including assessment design adequacy, criteria of quality, and validity of interpretations, in the context of the emerging research agenda in the field.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexander, P. A., & Judy, J. E. (1988). The interaction of domain-specific and strategic knowledge in academic performance. Review of Education Research, 58(4), 375–404.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., et al. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review. Psychological Bulletin, 136(2), 151–173.
Anderson, L. W., & Krathwohl, D. R. (with Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C.). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Baker, E. L. (1997). Model-based performance assessment. Theory Into Practice, 36(4), 247–254.
Baker, E. L. (2007a, August/September). The end(s) of testing (2007 AERA presidential address). Educational Researcher, 36(6), 309–317. Retrieved October 2, 2007, from http://www.aera.net/uploadedFiles/Publications/Journals/Educational_Researcher/3606/09edr07_309-317.pdf.
Baker, E. L. (2007b). Model-based assessments to support learning and accountability: The evolution of CRESST’s research on multiple-purpose measures. Educational Assessment (Special Issue), 12(3&4), 179–194.
Baker, E. L. (2007c). Teacher use of formal assessment in the classroom. In W. D. Hawley with D. L. Rollie (Eds.), The keys to effective schools: Educational reform as continuous improvement (2nd ed., pp. 67–84). Thousand Oaks, CA: Corwin.
Baker, E. L., Chung, G. K. W. K., & Delacruz, G. C. (2012). The best and future uses of assessment in games. In M. C. Mayrath, J. Clarke-Midura, D. H. Robinson, & G. Schraw (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 227–246). Charlotte, NC: Information Age.
Baker, E. L., Freeman, M., & Clayton, S. (1991). Cognitive assessment of history for large-scale testing. In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 131–153). Englewood Cliffs, NJ: Prentice-Hall.
Baker, E. L., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers in Human Behavior, 15(3/4), 269–282.
Bloom, B. S. (with Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R.). (1956). Taxonomy of educational objectives: The classification of education goals. Handbook I: Cognitive domain. New York: Longman.
Brockhaus, R. H., Sr. (1980). Risk taking propensity of entrepreneurs. Academy of Management Journal, 23(3), 509–520.
Chung, G. K. W. K., & Baker, E. L. (1997). Year 1 technology studies: Implications for technology in assessment (CSE Tech. Rep. No. 459). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
Chung, G. K. W. K., Niemi, D., & Bewley, W. L. (2003, April). Assessment applications of ontologies. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
Cline, V. B., Croft, R. G., & Courrier, S. (1973). Desensitization of children to television violence. Journal of Personality and Social Psychology, 27, 360–365.
de Jong, T., & Ferguson-Hessler, M. G. M. (1996). Types and qualities of knowledge. Educational Psychologist, 31(2), 105–113.
Dragan, I., Tepper, K., & Misso, M. (2011). Teaching evidence based medicine literature searching skills to medical students during the clinical years—A protocol for a randomized controlled trial. BMC Medical Education, 11, 49. Retrieved from http://www.biomedcentral.com/content/pdf/1472-6920-11-49.pdf.
Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37, 65–84.
Feltovich, P. J., Spiro, R. J., Coulson, R. L., & Feltovich, J. (1996). Collaboration within and among minds: Mastering complexity, individually and in groups. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm (pp. 25–44). Mahwah, NJ: Erlbaum.
Fletcher, J. D. (2004). Cognitive readiness: Preparing for the unexpected. Alexandria, VA: Institute for Defense Analyses. Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA458683&Location=U2&doc=GetTRDoc.pdf
Graham, S. (1991). A review of attribution theory in achievement contexts. Educational Psychology Review, 3(1), 5–39.
Guess, D. (1969). A functional analysis of receptive language and productive speech: Acquisition of the plural morpheme. Journal of Applied Behavior Analysis, 2(1), 55–65.
Hargie, O. (2006). The handbook of communication skills (3rd ed.). New York: Routledge.
Iseli, M. (2011). Ontology development: Overview and example (Draft CRESST Whitepaper). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
Iseli, M. R., Koenig, A. D., Lee, J. J., & Wainess, R. (2010). Automatic assessment of complex task performance in games and simulations (CRESST Report 775). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Johnson-Laird, P. (2006). How we reason. New York: Oxford University Press.
Keltner, D. (2004). The compassionate instinct. Greater good. Retrieved from http://greatergood.berkeley.edu/article/item/the_compassionate_instinct/
Koenig, A. D., Lee, J., Iseli, M., & Wainess, R. (2009, November). A conceptual framework for assessing performance in games and simulations. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, FL.
Lee, J. J., Bewley, W. L., Jones, B., Min, H., & Kang, T. (2009, November). Assessing performance in a simulated combat information center. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, FL.
Lee, J. J., Jones, B., & Min, H. (2009, April). Assessing performance in the multi-mission team trainer. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Linz, D., Donnerstein, E., & Penrod, S. (1988). Effects of long-term exposure to violent and sexually degrading depictions of women. Journal of Personality and Social Psychology, 55, 758–768.
Logel, C. R., Walton, G. M., Spencer, S. J., Peach, J., & Mark, Z. P. (2012). Unleashing latent ability: Implications of stereotype threat for college admissions. Educational Psychologist, 47(1), 42–50.
Mayer, S. (2010). Problem solving ontology (CRESST Whitepaper). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (pp. 13–103). New York: McMillan.
National Governors Association Center for Best Practices, Council of Chief State School Officers. (2010). Common core state standards. Washington, DC: National Governors Association Center for Best Practices, Council of Chief State School Officers. http://www.corestandards.org/the-standards
O’Neil, H. F., Chuang, S., & Chung, G. K. W. K. (2003). Issues in the computer-based assessment of collaborative problem solving. Assessment in Education, 10, 361–373.
O’Neil, H. F., Jr., Chung, G. K. W. K., & Brown, R. (1997). Use of networked simulations as a context to measure team competencies. In H. F. O’Neil Jr. (Ed.), Workforce readiness: Competencies and assessment (pp. 411–452). Mahwah, NJ: Erlbaum.
O’Neil, H. F., Wang, S.-L., Chung, G. K. W. K., & Herl, H. E. (2000). Assessment of teamwork skills using computer-based teamwork simulations. In H. F. O’Neil & D. H. Andrews (Eds.), Aircrew training and assessment (pp. 245–276). Mahwah, NJ: Erlbaum.
O’Neil, H. F., Jr., Wang, S., Lee, C., Mulkey, J., & Baker, E. L. (2003). Assessment of teamwork skills via a teamwork questionnaire. In H. F. O’Neil Jr. & R. S. Perez (Eds.), Technology applications in education: A learning view (pp. 283–303). Mahwah, NJ: Erlbaum.
Paul, G. L., & Bernstein, D. A. (1976). Anxiety and clinical problems: Systematic desensitization and related techniques. In J. T. Spence, R. C. Carson, & J. W. Thibaut (Eds.), Behavioral approaches to therapy. Morristown, NJ: General Learning Press.
Perry, R. P., Stupnisky, R. H., Hall, N. C., Chipperfield, J. G., & Weiner, B. (2010). Bad starts and better finishes: Attributional retraining and initial performance in competitive achievement settings. Journal of Social and Clinical Psychology, 29(6), 668–700.
Phelan, J., Dai, Y., Valderrama, M., & Herman, J. (2011). Development of learning-based assessments in literacy: Towards the goal of college readiness (Whitepaper submitted to 2012 AERA). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Roger, D., & Nesshoever, W. (1987). The construction and preliminary validation of a scale for measuring emotional control. Personality and Individual Differences, 8(4), 527–534.
Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471–499.
Steele, C. M., Spencer, S. J., & Lynch, M. (1993). Self-image resilience and dissonance: The role of affirmational resources. Journal of Personality and Social Psychology, 64(6), 885–896.
Sweller, J. (2003). Evolution of human cognitive architecture. In B. Ross (Ed.), The psychology of learning and motivation (Vol. 43, pp. 215–266). San Diego, CA: Academic.
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Webb, N. M. (1985). Student interaction and learning in small groups: A research summary. In R. Slavin, S. Sharan, S. Kagan, R. H. Lazarowitz, C. Webb, & R. Schmuck (Eds.), Learning to cooperate, cooperating to learn (pp. 147–172). New York: Plenum.
Weiner, B., Graham, S., & Reyna, C. (1997). An attributional examination of retributive versus utilitarian philosophies of punishment. Social Justice Research, 10, 431–452.
Acknowledgments
The work reported herein was supported under the Office of Naval Research, PR/Award Number N00014-09-C-0813; the Defense Advanced Research Projects Agency, PR/Award Number N00014-12-C-0090; and the Educational Research and Development Centers Program, PR/Award Number R305C080015, as administered by the Institute of Education Sciences, the U.S. Department of Education. The findings and opinions expressed in this chapter do not necessarily reflect the positions or policies of the Office of Naval Research, the Defense Advanced Research Projects Agency, or the Institute of Education Sciences, the U.S. Department of Education.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this chapter
Cite this chapter
Baker, E.L. (2014). Learning and Assessment: Twenty-First Century Skills and Cognitive Readiness. In: O'Neil, H., Perez, R., Baker, E. (eds) Teaching and Measuring Cognitive Readiness. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-7579-8_3
Download citation
DOI: https://doi.org/10.1007/978-1-4614-7579-8_3
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4614-7578-1
Online ISBN: 978-1-4614-7579-8
eBook Packages: Humanities, Social Sciences and LawEducation (R0)