Skip to main content
SpringerLink
Log in

A conceptual analysis of five measures of metacognitive monitoring

  • Published:
Metacognition and Learning Aims and scope Submit manuscript

Abstract

This paper describes five indices of metacognitive monitoring referred to as absolute accuracy, relative accuracy, bias, scatter, and discrimination. I provide definitions, formulae, and a discussion of the underlying construct that each of the five types of scores measures. I discuss the type of information provided by each measure and compare situations in which each measure is most appropriate. Recommendations are made for best measurement practice, as well as directions for future research. Recommendations focus on providing an operational definition of the construct being measured, selecting the most appropriate outcome measure, and using multiple measures whenever possible to triangulate findings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Agresti, A. (1984). Analysis of ordinal categorical data. New York: Wiley.

    Google Scholar 

  • Agresti, A. (1990). Categorical data analysis. New York: Wiley.

    Google Scholar 

  • Allwood, C. M., Jonsson, A. C., & Granhag, P. A. (2005). The effects of source and type of feedback on child witnesses’ metamemory accuracy. Applied Cognitive Psychology, 19, 331–344. doi:10.1002/acp.1071.

    Article  Google Scholar 

  • Bol, L., Hacker, D., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. Journal of Experimental Education, 73, 269–290. doi:10.3200/JEXE.73.4.269-290.

    Article  Google Scholar 

  • Burson, K. A., Larrick, R. P., & Klayman, J. (2006). Skilled or unskilled, but still unaware of it: Perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90, 60–77. doi:10.1037/0022-3514.90.1.60.

    Article  Google Scholar 

  • Butler, D., & Winne, P. (1995). Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65, 245–281.

    Google Scholar 

  • Connor, L. T., Dunlosky, J., & Hertzog, C. (1997). Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging, 12, 50–71. doi:10.1037/0882-7974.12.1.50.

    Article  Google Scholar 

  • Dougherty, M. R. P., & Sprenger, A. (2006). The influence of improper sets of information of judgment: How irrelevant information can bias judged probability. Journal of Experimental Psychology. General, 135, 262–281. doi:10.1037/0096-3445.135.2.262.

    Article  Google Scholar 

  • Dunlosky, J., Rawson, K. A., & Middleton, E. L. (2005). What constrains the accuracy of metacomprehension judgments? Testing the transfer-appropriate-monitoring and accessibility hypothesis. Journal of Memory and Language, 52, 551–565. doi:10.1016/j.jml.2005.01.011.

    Article  Google Scholar 

  • Dunn, J. C. (2004). Remember-know: A matter of confidence. Psychological Review, 111, 524–542. doi:10.1037/0033-295X.111.2.524.

    Article  Google Scholar 

  • Flavell, J. H. (1979). Metacognition and cognitive monitoring. The American Psychologist, 34, 906–911. doi:10.1037/0003–066X.34.10.906.

    Article  Google Scholar 

  • Flavell, J. H., & Wellman, H. (1977). Metamemory. In R. V. Vail, & J. W. Hagen (Eds.), Perspectives on the development of memory and cognition (pp. 220–241). Hillsdale: Erlbaum.

    Google Scholar 

  • Gillstrom, A., & Ronnberg, J. (1995). Comprehension calibration and recall prediction accuracy of texts: Reading skill, reading strategies, and effort. Journal of Educational Psychology, 87, 545–558. doi:10.1037/0022-0663.87.4.545.

    Article  Google Scholar 

  • Glenberg, A. M, & Epstein, W. (1987). Inexpert calibration of comprehension. Memory & Cognition, 15(1), 84–93.

    Google Scholar 

  • Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classification. Journal of the American Statistical Association, 49, 732–764. doi:10.2307/2281536.

    Article  Google Scholar 

  • Juslin, P., Olsson, N., & Winman, A. (1996). Calibration and diagnosticity of confidence in eyewitness testimony: comments on what can be inferred from the low confidence-accuracy correlation. Journal of Experimental Psychology. Learning, Memory, and Cognition, 22, 1304–1316. doi:10.1037/0278-7393.22.5.1304.

    Article  Google Scholar 

  • Keleman, W. L., Frost, P. J., & Weaver, C. A. (2000). Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition, 28, 92–107.

    Google Scholar 

  • Keren, G. (1991). Calibration and probability judgments: conceptual and methodological issues. Acta Psychologica, 77, 217–273. doi:10.1016/0001-6918(91)90036-Y.

    Article  Google Scholar 

  • Koriat, A., Sheffer, L., & Ma’ayan, H. (2002). Comparing objective and subjective learning curves: Judgments of learning exhibit increased overconfidence with practice. Journal of Experimental Psychology. General, 131, 147–162. doi:10.1037/0096-3445.131.2.147.

    Article  Google Scholar 

  • Lin, L. M., & Zabrucky, K. M. (1998). Calibration of comprehension: Research and implications for education and instruction. Contemporary Educational Psychology, 23, 345–391. doi:10.1006/ceps.1998.0972.

    Article  Google Scholar 

  • Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97, 723–731. doi:10.1037/0022-0663.97.4.723.

    Article  Google Scholar 

  • McCormick, C. B. (2003). Metacognition and learning. In W. M. Reynolds, & G. E. Miller (Eds.), Handbook of psychology: Educational psychology (pp. 79–102). Hoboken: Wiley.

    Google Scholar 

  • Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not the absolute performance on an individual item: comments on Schraw (1995). Applied Cognitive Psychology, 10, 257–260. doi:10.1002/(SICI)1099-0720(199606)10:3<257::AID-ACP400>3.0.CO;2-9.

    Article  Google Scholar 

  • Nelson, T. O., Narens, L., & Dunlosky, J. (2004). A revised methodology for research in metamemory: Pre-judgment recall and monitoring (PRAM). Psychological Methods, 9, 53–69. doi:10.1037/1082-989X.9.1.53.

    Article  Google Scholar 

  • Nietfeld, J. L., Cao, L., & Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom. Journal of Experimental Education, 74, 7–28.

    Google Scholar 

  • Nietfeld, J. L., Enders, C. K., & Schraw, G. (2006). A Monte Carlo comparison of measures of relative and absolute monitoring accuracy. Educational and Psychological Measurement, 66, 258–271. doi:10.1177/0013164404273945.

    Article  Google Scholar 

  • Osterlind, S. J. (2006). Modern measurement: Theory, practice, and applications of mental appraisal. Upper Saddle River: Prentice Hall.

    Google Scholar 

  • Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego: Academic.

    Chapter  Google Scholar 

  • Prins, F. J., Veenman, M. V. J., & Elshout, J. J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory. Learning and Instruction, 16, 374–387. doi:10.1016/j.learninstruc.2006.07.008.

    Article  Google Scholar 

  • Romesburg, H. C. (1984). Cluster analysis for researchers. London: Wadsworth.

    Google Scholar 

  • Schunk, D. H., & Zimmerman, B. J. (2006). Competence and control beliefs: Distinguishing the means and the ends. In P. Alexander, & P. Winne (Eds.), Handbook of educational psychology (pp. 349–367, 2nd ed.). San Diego: Academic.

    Google Scholar 

  • Siegel, S., & Castellan Jr., N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Souchay, C., Isingrini, M., Clarys, D., Taconnat, L., & Eustache, F. (2004). Executive functioning and judgment-of-learning versus feeling-of-knowing in older adults. Experimental Aging Research, 30, 47–62. doi:10.1080/03610730490251478.

    Article  Google Scholar 

  • Stankov, L. (1998). Calibration curves, scatterplots, and the distinction between general knowledge and perceptual tasks. Learning and Individual Differences, 10, 104–120. doi:10.1016/S1041-6080(99)80141-1.

    Article  Google Scholar 

  • Stewart, N., Brown, G. D., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112, 881–911. doi:10.1037/0033-295X.112.4.881.

    Article  Google Scholar 

  • Thiede, K. W., & Anderson, M. C. (2003). Summarizing can improve metacomprehension accuracy. Contemporary Educational Psychology, 28, 129–160. doi:10.1016/S0361-476X(02)00011-5.

    Article  Google Scholar 

  • Thiede, K. W., Dunlosky, J., Griffin, D., & Wiley, J. (2005). Understanding the delayed-keyword effect on metacomprehension accuracy. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31, 1267–1280. doi:10.1037/0278-7393.31.6.1267.

    Article  Google Scholar 

  • Thompson, W. B. (1999). Individual differences in memory monitoring accuracy. Learning and Individual Differences, 11, 365–376. doi:10.1016/S1041-6080(99)80009-0.

    Article  Google Scholar 

  • Thompson, W. B., & Mason, S. E. (1996). Instability of individual differences in the association between confidence judgments and memory performance. Memory & Cognition, 94, 226–234.

    Google Scholar 

  • Veenman, M. V. J., Van Hout-Walters, B. H. A., & Afflerbach, P. (2006). Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 1, 3–14. doi:10.1007/s11409-006-6893-0.

    Article  Google Scholar 

  • Wallsten, T. S. (1996). Commentary: an analysis of judgment research analyses. Organizational Behavior and Human Decision Processes, 65, 220–226. doi:10.1006/obhd.1996.0022.

    Article  Google Scholar 

  • Wallsten, T. S., & Gu, H. (2003). Distinguishing choice and subjective probability estimation processes: implications for theories of judgment and for cross-cultural comparisons. Organizational Behavior and Decision Processes, 90, 111–123. doi:10.1016/S0749-5978(02)00513-7.

    Article  Google Scholar 

  • Weaver, C. A. (1990). Constraining factors in the calibration of comprehension. Journal of Experimental Psychology. Learning, Memory, and Cognition, 16, 214–222. doi:10.1037/0278-7393.16.2.214.

    Article  Google Scholar 

  • White, B., & Frederiksen, J. (2005). A theoretical framework and approach for fostering metacognitive development. Educational Psychologist, 40, 211–223. doi:10.1207/s15326985ep4004_3.

    Article  Google Scholar 

  • Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman, & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153–189). Mahwah: Erlbaum.

    Google Scholar 

  • Wright, D. B. (1996). Measuring feeling of knowing: comment on Schraw (1995). Applied Cognitive Psychology, 10, 261–268. doi:10.1002/(SICI)1099-0720(199606)10:3<261::AID-ACP387>3.0.CO;2-0.

    Article  Google Scholar 

  • Yates, J. F. (1988). Analyzing the accuracy of probability judgments for multiple events: an extension of the covariance decomposition. Organizational Behavior and Human Decision Processes, 41, 281–299. doi:10.1016/0749-5978(88)90031-3.

    Article  Google Scholar 

  • Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs: Prentice-Hall.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Educational Psychology, University of Nevada, Las Vegas, 4505 Maryland Parkway, P.O. Box 453003, Las Vegas, NV, 89154, USA

    Gregory Schraw

Authors
  1. Gregory Schraw
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gregory Schraw.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schraw, G. A conceptual analysis of five measures of metacognitive monitoring. Metacognition Learning 4, 33–45 (2009). https://doi.org/10.1007/s11409-008-9031-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11409-008-9031-3

Keywords

Search

Navigation