Advertisement

Metacognition and Learning

, Volume 4, Issue 1, pp 33–45 | Cite as

A conceptual analysis of five measures of metacognitive monitoring

  • Gregory SchrawEmail author
Article

Abstract

This paper describes five indices of metacognitive monitoring referred to as absolute accuracy, relative accuracy, bias, scatter, and discrimination. I provide definitions, formulae, and a discussion of the underlying construct that each of the five types of scores measures. I discuss the type of information provided by each measure and compare situations in which each measure is most appropriate. Recommendations are made for best measurement practice, as well as directions for future research. Recommendations focus on providing an operational definition of the construct being measured, selecting the most appropriate outcome measure, and using multiple measures whenever possible to triangulate findings.

Keywords

Metacognitive monitoring Accuracy Bias Other measures 

References

  1. Agresti, A. (1984). Analysis of ordinal categorical data. New York: Wiley.Google Scholar
  2. Agresti, A. (1990). Categorical data analysis. New York: Wiley.Google Scholar
  3. Allwood, C. M., Jonsson, A. C., & Granhag, P. A. (2005). The effects of source and type of feedback on child witnesses’ metamemory accuracy. Applied Cognitive Psychology, 19, 331–344. doi: 10.1002/acp.1071.CrossRefGoogle Scholar
  4. Bol, L., Hacker, D., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. Journal of Experimental Education, 73, 269–290. doi: 10.3200/JEXE.73.4.269-290.CrossRefGoogle Scholar
  5. Burson, K. A., Larrick, R. P., & Klayman, J. (2006). Skilled or unskilled, but still unaware of it: Perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90, 60–77. doi: 10.1037/0022-3514.90.1.60.CrossRefGoogle Scholar
  6. Butler, D., & Winne, P. (1995). Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65, 245–281.Google Scholar
  7. Connor, L. T., Dunlosky, J., & Hertzog, C. (1997). Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging, 12, 50–71. doi: 10.1037/0882-7974.12.1.50.CrossRefGoogle Scholar
  8. Dougherty, M. R. P., & Sprenger, A. (2006). The influence of improper sets of information of judgment: How irrelevant information can bias judged probability. Journal of Experimental Psychology. General, 135, 262–281. doi: 10.1037/0096-3445.135.2.262.CrossRefGoogle Scholar
  9. Dunlosky, J., Rawson, K. A., & Middleton, E. L. (2005). What constrains the accuracy of metacomprehension judgments? Testing the transfer-appropriate-monitoring and accessibility hypothesis. Journal of Memory and Language, 52, 551–565. doi: 10.1016/j.jml.2005.01.011.CrossRefGoogle Scholar
  10. Dunn, J. C. (2004). Remember-know: A matter of confidence. Psychological Review, 111, 524–542. doi: 10.1037/0033-295X.111.2.524.CrossRefGoogle Scholar
  11. Flavell, J. H. (1979). Metacognition and cognitive monitoring. The American Psychologist, 34, 906–911. doi: 10.1037/0003–066X.34.10.906.CrossRefGoogle Scholar
  12. Flavell, J. H., & Wellman, H. (1977). Metamemory. In R. V. Vail, & J. W. Hagen (Eds.), Perspectives on the development of memory and cognition (pp. 220–241). Hillsdale: Erlbaum.Google Scholar
  13. Gillstrom, A., & Ronnberg, J. (1995). Comprehension calibration and recall prediction accuracy of texts: Reading skill, reading strategies, and effort. Journal of Educational Psychology, 87, 545–558. doi: 10.1037/0022-0663.87.4.545.CrossRefGoogle Scholar
  14. Glenberg, A. M, & Epstein, W. (1987). Inexpert calibration of comprehension. Memory & Cognition, 15(1), 84–93.Google Scholar
  15. Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classification. Journal of the American Statistical Association, 49, 732–764. doi: 10.2307/2281536.CrossRefGoogle Scholar
  16. Juslin, P., Olsson, N., & Winman, A. (1996). Calibration and diagnosticity of confidence in eyewitness testimony: comments on what can be inferred from the low confidence-accuracy correlation. Journal of Experimental Psychology. Learning, Memory, and Cognition, 22, 1304–1316. doi: 10.1037/0278-7393.22.5.1304.CrossRefGoogle Scholar
  17. Keleman, W. L., Frost, P. J., & Weaver, C. A. (2000). Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition, 28, 92–107.Google Scholar
  18. Keren, G. (1991). Calibration and probability judgments: conceptual and methodological issues. Acta Psychologica, 77, 217–273. doi: 10.1016/0001-6918(91)90036-Y.CrossRefGoogle Scholar
  19. Koriat, A., Sheffer, L., & Ma’ayan, H. (2002). Comparing objective and subjective learning curves: Judgments of learning exhibit increased overconfidence with practice. Journal of Experimental Psychology. General, 131, 147–162. doi: 10.1037/0096-3445.131.2.147.CrossRefGoogle Scholar
  20. Lin, L. M., & Zabrucky, K. M. (1998). Calibration of comprehension: Research and implications for education and instruction. Contemporary Educational Psychology, 23, 345–391. doi: 10.1006/ceps.1998.0972.CrossRefGoogle Scholar
  21. Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97, 723–731. doi: 10.1037/0022-0663.97.4.723.CrossRefGoogle Scholar
  22. McCormick, C. B. (2003). Metacognition and learning. In W. M. Reynolds, & G. E. Miller (Eds.), Handbook of psychology: Educational psychology (pp. 79–102). Hoboken: Wiley.Google Scholar
  23. Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not the absolute performance on an individual item: comments on Schraw (1995). Applied Cognitive Psychology, 10, 257–260. doi: 10.1002/(SICI)1099-0720(199606)10:3<257::AID-ACP400>3.0.CO;2-9.CrossRefGoogle Scholar
  24. Nelson, T. O., Narens, L., & Dunlosky, J. (2004). A revised methodology for research in metamemory: Pre-judgment recall and monitoring (PRAM). Psychological Methods, 9, 53–69. doi: 10.1037/1082-989X.9.1.53.CrossRefGoogle Scholar
  25. Nietfeld, J. L., Cao, L., & Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom. Journal of Experimental Education, 74, 7–28.Google Scholar
  26. Nietfeld, J. L., Enders, C. K., & Schraw, G. (2006). A Monte Carlo comparison of measures of relative and absolute monitoring accuracy. Educational and Psychological Measurement, 66, 258–271. doi: 10.1177/0013164404273945.CrossRefGoogle Scholar
  27. Osterlind, S. J. (2006). Modern measurement: Theory, practice, and applications of mental appraisal. Upper Saddle River: Prentice Hall.Google Scholar
  28. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego: Academic.CrossRefGoogle Scholar
  29. Prins, F. J., Veenman, M. V. J., & Elshout, J. J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory. Learning and Instruction, 16, 374–387. doi: 10.1016/j.learninstruc.2006.07.008.CrossRefGoogle Scholar
  30. Romesburg, H. C. (1984). Cluster analysis for researchers. London: Wadsworth.Google Scholar
  31. Schunk, D. H., & Zimmerman, B. J. (2006). Competence and control beliefs: Distinguishing the means and the ends. In P. Alexander, & P. Winne (Eds.), Handbook of educational psychology (pp. 349–367, 2nd ed.). San Diego: Academic.Google Scholar
  32. Siegel, S., & Castellan Jr., N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.Google Scholar
  33. Souchay, C., Isingrini, M., Clarys, D., Taconnat, L., & Eustache, F. (2004). Executive functioning and judgment-of-learning versus feeling-of-knowing in older adults. Experimental Aging Research, 30, 47–62. doi: 10.1080/03610730490251478.CrossRefGoogle Scholar
  34. Stankov, L. (1998). Calibration curves, scatterplots, and the distinction between general knowledge and perceptual tasks. Learning and Individual Differences, 10, 104–120. doi: 10.1016/S1041-6080(99)80141-1.CrossRefGoogle Scholar
  35. Stewart, N., Brown, G. D., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112, 881–911. doi: 10.1037/0033-295X.112.4.881.CrossRefGoogle Scholar
  36. Thiede, K. W., & Anderson, M. C. (2003). Summarizing can improve metacomprehension accuracy. Contemporary Educational Psychology, 28, 129–160. doi: 10.1016/S0361-476X(02)00011-5.CrossRefGoogle Scholar
  37. Thiede, K. W., Dunlosky, J., Griffin, D., & Wiley, J. (2005). Understanding the delayed-keyword effect on metacomprehension accuracy. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31, 1267–1280. doi: 10.1037/0278-7393.31.6.1267.CrossRefGoogle Scholar
  38. Thompson, W. B. (1999). Individual differences in memory monitoring accuracy. Learning and Individual Differences, 11, 365–376. doi: 10.1016/S1041-6080(99)80009-0.CrossRefGoogle Scholar
  39. Thompson, W. B., & Mason, S. E. (1996). Instability of individual differences in the association between confidence judgments and memory performance. Memory & Cognition, 94, 226–234.Google Scholar
  40. Veenman, M. V. J., Van Hout-Walters, B. H. A., & Afflerbach, P. (2006). Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 1, 3–14. doi: 10.1007/s11409-006-6893-0.CrossRefGoogle Scholar
  41. Wallsten, T. S. (1996). Commentary: an analysis of judgment research analyses. Organizational Behavior and Human Decision Processes, 65, 220–226. doi: 10.1006/obhd.1996.0022.CrossRefGoogle Scholar
  42. Wallsten, T. S., & Gu, H. (2003). Distinguishing choice and subjective probability estimation processes: implications for theories of judgment and for cross-cultural comparisons. Organizational Behavior and Decision Processes, 90, 111–123. doi: 10.1016/S0749-5978(02)00513-7.CrossRefGoogle Scholar
  43. Weaver, C. A. (1990). Constraining factors in the calibration of comprehension. Journal of Experimental Psychology. Learning, Memory, and Cognition, 16, 214–222. doi: 10.1037/0278-7393.16.2.214.CrossRefGoogle Scholar
  44. White, B., & Frederiksen, J. (2005). A theoretical framework and approach for fostering metacognitive development. Educational Psychologist, 40, 211–223. doi: 10.1207/s15326985ep4004_3.CrossRefGoogle Scholar
  45. Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman, & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153–189). Mahwah: Erlbaum.Google Scholar
  46. Wright, D. B. (1996). Measuring feeling of knowing: comment on Schraw (1995). Applied Cognitive Psychology, 10, 261–268. doi: 10.1002/(SICI)1099-0720(199606)10:3<261::AID-ACP387>3.0.CO;2-0.CrossRefGoogle Scholar
  47. Yates, J. F. (1988). Analyzing the accuracy of probability judgments for multiple events: an extension of the covariance decomposition. Organizational Behavior and Human Decision Processes, 41, 281–299. doi: 10.1016/0749-5978(88)90031-3.CrossRefGoogle Scholar
  48. Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs: Prentice-Hall.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Educational PsychologyUniversity of Nevada, Las VegasLas VegasUSA

Personalised recommendations