Abstract
In this theoretical paper, I present a short critical review of research on calibration. Based on this conceptual analysis I argue for two extensions of this construct: In addition to traditional applications, the methodology should be transferred to also measure calibration between students’ metacognitive control processes (traditionally, only students’ metacognitive judgments were considered) and important external criteria (traditionally, judgments were only compared to students’ own performance). As an illustrative example, one application context will be highlighted where these proposed extensions would alleviate potential problems with the traditional conceptualization: While students’ idiosyncratic task definitions constitute unwanted error variance in the traditional account it would be possible to investigate them as a primary research question within this extended notion. More specifically, it would be possible to investigate how well students’ learning processes match objective task demands. I will put forward theoretical and empirical arguments in favor of these suggestions.
Similar content being viewed by others
References
Allwood, C. M., Jonsson, A.-C., & Granhag, P. A. (2005). The effects of source and type of feedback on child witnesses’ metamemory accuracy. Applied Cognitive Psychology, 19, 331–344. doi:10.1002/acp.1071.
Anderson, G., & Beal, C. R. (1995). Children’s recognition of inconsistencies in science texts: Multiple measures of comprehension monitoring. Applied Cognitive Psychology, 9, 261–272. doi:10.1002/acp.2350090307.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching, and assessing. A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Artelt, C. (2000). Wie prädiktiv sind retrospektive Selbstberichte über den Gebrauch von Lernstrategien für strategisches Lernen. Zeitschrift fur Padagogische Psychologie, 14(2/3), 72–84 How predictive are retrospective self-reports about the use of learning strategies. doi:10.1024//1010-0652.14.23.72.
Baker, L. (1984a). Spontaneous versus instructed use of multiple standards for evaluating comprehension: Effects of age, reading proficiency, and type of standard. Journal of Experimental Child Psychology, 38, 289–311. doi:10.1016/0022-0965(84)90127-9.
Baker, L. (1984b). Children’s effective use of multiple standards for evaluating their comprehension. Journal of Educational Psychology, 76(4), 588–597. doi:10.1037/0022-0663.76.4.588.
Baker, L. (1985a). Differences in the standards used by college students to evaluate their comprehension of expository prose. Reading Research Quarterly, 20(3), 297–313. doi:10.2307/748020.
Baker, L. (1985b). How do we know when we don’t understand? Standards for evaluating text comprehension. In D. L. Forrest-Pressley, G. E. MacKinnon, & T. G. Waller (Eds.), Metacognition, cognition, and human performance (pp. 155–205). Orlando: Academic.
Bannert, M. (2003). Assessment of metacognitive skills by means of thinking-aloud instruction and reflection prompts. Does the method affect the learning performance? Paper presented at the Biannual conference of the European Association of Research in Learning and Instruction (EARLI), Padua, Italy.
Barnett, J. E., & Hixon, J. E. (1997). Effects of grade level and subject on student test score predictions. The Journal of Educational Research, 90(3), 170–174.
Beal, C. R. (1996). The role of comprehension monitoring in children’s revisions. Educational Psychology Review, 8(3), 219–238. doi:10.1007/BF01464074.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook I: cognitive domain. New York: David McKay.
Bol, L., Hacker, D. J., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. Journal of Experimental Education, 73(4), 269–290. doi:10.3200/JEXE.73.4.269-290.
Broekkamp, H., van den Bergh, H., van Hout-Wolters, B. H. A. M., & Rijlaardam, G. (2002a). Will that be on the test? Perceived task demands and test performance in a classroom context. European Journal of Psychology of Education, XVII(1), 75–92.
Broekkamp, H., van Hout-Wolters, B. H. A. M., Rijlaardam, G., & van den Bergh, H. (2002b). Importance in instructional text: teachers’ and students’ perception of task demands. Journal of Educational Psychology, 94(2), 260–271. doi:10.1037/0022-0663.94.2.260.
Broekkamp, H., van Hout-Wolters, B. H. A. M., van den Bergh, H., & Rijlaardam, G. (2004). Teachers’ task demands, students’ test expectations, and actual test content. The British Journal of Educational Psychology, 74, 205–220. doi:10.1348/000709904773839842.
Burson, K. A., Klayman, J., & Larrick, R. P. (2006). Skilled or unskilled, but still unaware of it: How perception of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90(1), 60–77. doi:10.1037/0022-3514.90.1.60.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281.
Carver, C. S., & Scheier, M. (1990). Principles of self-regulation: Action and emotion. In E. T. Higgins, & R. M. Sorrentino (Eds.), Handbook of motivation and cognition: Foundations of social behavior (vol. 2). New York, NY: Guilford.
Dougherty, M. R., & Sprenger, A. (2006). The influence of imporper sets of information on judgment: How irrelevant information can bias judged probability. Journal of Experimental Psychology. General, 135(2), 262–281. doi:10.1037/0096-3445.135.2.262.
Dunlosky, J., & Hertzog, C. (2000). Updating knowledge about encoding strategies: A componential analysis of learning about strategy effectiveness from task experience. Psychology and Aging, 15(3), 462–474. doi:10.1037/0882-7974.15.3.462.
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87. doi:10.1111/1467-8721.01235.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. doi:10.1037/0033-295X.87.3.215.
Fitts, P. M. (1964). Perceptual-motor skill learning. In A. W. Melton (Ed.), Categories of human learning (pp. 243–385). New York: Academic.
Fitzgerald, J. T., White, C. B., & Gruppen, L. D. (2003). A longitudinal study of self-assessment accuracy. Medical Education, 37, 645–649. doi:10.1046/j.1365-2923.2003.01567.x.
Garavalia, L. S., & Gredler, M. E. (2003). An exploratory study of academic goal setting, achievement calibration and self-regulated learning. Journal of Instructional Psychology, 29(4), 221–230.
García-Arista, E., Campanario, J. M., & Otero, J. (1996). Influences of subject matter setting on comprehension monitoring. European Journal of Psychology of Education, XI(4), 427–441.
Garner, R. (1980). Monitoring of understanding: An investigation of good and poor readers’ awareness of induced miscomprehension of text. Journal of Reading Behavior, XII(1), 55–64.
Gasser, M., & Tan, R. (2005). Performance estimates and confidence calibration for a perceptual-motor task. North American Journal of Psychology, 7(3), 457–468.
Glenberg, A. M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology. Learning, Memory, and Cognition, 11(4), 702–718. doi:10.1037/0278-7393.11.1-4.702.
Glenberg, A. M., & Epstein, W. (1987). Inexpert calibration of comprehension. Memory & Cognition, 15(1), 84–93.
Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental Psychology. General, 116(2), 119–136. doi:10.1037/0096-3445.116.2.119.
Goodman, L. A., & Kruskal, W. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732–769. doi:10.2307/2281536.
Hacker, D. J. (1997). Comprehension monitoring of written discourse across early-to-middle adolescence. Reading and Writing: An Interdisciplinary Journal, 9, 207–240. doi:10.1023/A:1007989901667.
Hacker, D. J. (1998). Self-regulated comprehension during normal reading. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 165–191). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Hacker, D. J., Bol, L., Horgan, D. D., & Rakow, E. A. (2000). Test prediction and performance in a classroom context. Journal of Educational Psychology, 92(1), 160–170. doi:10.1037/0022-0663.92.1.160.
Hadwin, A. F., Oshige, M., Fior, M. N., Tupper, K., & Miller, M. F. W. (2008). Examining the agreement between students and instructor task perceptions in a complex engineering design task. Paper presented at the Annual Meeting of the American Educational Research Association (AERA), New York.
Hadwin, A. F., Winne, P. H., Stockley, D. B., Nesbit, J. C., & Woszczyna, C. (2001). Context moderates students’ self-reports about how they study. Journal of Educational Psychology, 93(3), 477–487. doi:10.1037/0022-0663.93.3.477.
Hare, V. C., & Borchardt, K. M. (1985). Good and poor comprehenders’ detection of errors revisited. The Journal of Educational Research, 78(4), 237–241.
Hasselhorn, M. (1998). Metakognition [Metacognition]. In D. E. Rost (Ed.), Handwörterbuch der Pädagogischen Psychologie (pp. 348–351). Weinheim: Beltz.
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67(1), 88–140.
Howie, P., & Roebers, C. M. (2007). Developmental progression in the confidence-accuracy relationship in event recall: Insights provided by a calibration perspective. Applied Cognitive Psychology, 21, 871–893. doi:10.1002/acp.1302.
Jamieson-Noel, D., & Winne, P. H. (2003). Comparing self-reports to traces of studying behavior as representations of students’ studying and achievement. Zeitschrift fur Padagogische Psychologie, 17(3/4), 159–171. doi:10.1024//1010-0652.17.34.159.
Jonsson, A.-C., & Allwood, C. M. (2003). Stability and variability in the realism of confidence judgments over time, content domain, and gender. Personality and Individual Differences, 34, 559–574. doi:10.1016/S0191-8869(02)00028-4.
Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. The Journal of Applied Psychology, 74(4), 657–690. doi:10.1037/0021-9010.74.4.657.
Koriat, A., & Levy-Sardot, R. (1999). Processes underlying metacognitive judgments: Information-based and experience-based monitoring of one’s own knowledge. In S. Chaiken, & Y. Trope (Eds.), Dual-process theories in social psychology (pp. 483–502). New York: Guilford.
Kröner, S., & Biermann, A. (2007). The relationship between confidence and self-concept—Towards a model of response confidence. Intelligence, 35(6), 580–590. doi:10.1016/j.intell.2006.09.009.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134. doi:10.1037/0022-3514.77.6.1121.
Lin, L.-M., & Zabrucky, K. (1998). Calibration of comprehension: Research and implications for education and instruction. Contemporary Educational Psychology, 23, 345–391. doi:10.1006/ceps.1998.0972.
Lodewyk, K. R., & Winne, P. H. (2005). Relations among the structure of learning tasks, achievement, and changes in self-efficacy in secondary students. Journal of Educational Psychology, 97(1), 3–12. doi:10.1037/0022-0663.97.1.3.
Luwel, K., Verschaffel, L., Onghena, P., & De Corte, E. (2003). Analysing the adaptiveness of strategy choices using the choice/no-choice method: The case of numerosity judgments. The European Journal of Cognitive Psychology, 15(4), 511–537. doi:10.1080/09541440244000292.
Maki, R. H. (1998). Test predictions over text material. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in theory and practice (pp. 117–144). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Maki, R. H., & McGuire, M. J. (2002). Metacognition for text: Findings and implications for education. In T. J. Perfect, & B. L. Schwartz (Eds.), Applied metacognition. Cambridge: Cambridge University Press.
Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97(4), 723–731. doi:10.1037/0022-0663.97.4.723.
Maki, R. H., & Serra, M. (1992). Role of practice tests in the accuracy of test predictions on text material. Journal of Educational Psychology, 84(2), 200–217. doi:10.1037/0022-0663.84.2.200.
Moore, D., Lin-Agler, L. M., & Zabrucky, K. (2005). A source of metacomprehension inaccuracy. Reading Psychology, 26(3), 251–265. doi:10.1080/02702710590962578.
Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95(1), 109–133. doi:10.1037/0033-2909.95.1.109.
Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item. Comments on Schraw (1995). Applied Cognitive Psychology, 10, 257–260. doi:10.1002/(SICI)1099-0720(199606)10:3<257::AID-ACP400>3.0.CO;2-9.
Nelson, T. O. (1999). Cognition versus metacognition. In R. J. Sternberg (Ed.), The nature of cognition (pp. 625–641). Cambridge: MIT.
Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The “delayed-JOL effect”. Psychological Science, 2(4), 267–270. doi:10.1111/j.1467-9280.1991.tb00147.x.
Nelson, T. O., & Narens, L. (1994). Why investigate metacognition. In J. Metcalfe, & A. P. Shimamura (Eds.), Metacognition, knowing about knowing (pp. 1–25). Cambridge: MIT.
Nietfeld, J., Cao, L., & Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom. Journal of Experimental Education, 74(1), 7–28.
Nietfeld, J. L., Enders, C. K., & Schraw, G. (2006). A Monte Carlo comparison of measures of relative and absolute monitoring accuracy. Educational and Psychological Measurement, 66(2), 258–271. doi:10.1177/0013164404273945.
Nietfeld, J., & Schraw, G. (2002). The effect of knowledge and strategy training on monitoring accuracy. The Journal of Educational Research, 95(3), 131–142.
Otero, J., & Campanario, J. M. (1992). The relationship between academic achievement and metacognitive comprehension monitoring ability of Spanish secondary school students. Educational and Psychological Measurement, 52, 419–430. doi:10.1177/0013164492052002017.
Pallier, G. (2003). Gender differences in the self-assessment of accuracy on cognitive tasks. Sex Roles, 48(5/6), 265–276. doi:10.1023/A:1022877405718.
Pallier, G., Wilkinson, R., Danthiir, V., Kleitman, S., Knezevic, G., Stankov, L., et al. (2002). The role of individual differences in the accuracy of confidence judgments. The Journal of General Psychology, 129(3), 257–299.
Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385–407.
Plumb, C., Butterfield, E. C., Hacker, D. J., & Dunlosky, J. (1994). Error correction in text. Reading and Writing, 6(4), 347–360. doi:10.1007/BF01028848.
Pressley, M., & Ghatala, E. S. (1988). Delusions about performance on multiple-choice comprehension tests. Reading Research Quarterly, 24(4), 454–464. doi:10.2307/747643.
Rawson, K. A., & Dunlosky, J. (2002). Are performance predictions for text based on ease of processing. Journal of Experimental Psychology. Learning, Memory, and Cognition, 28(1), 69–80. doi:10.1037/0278-7393.28.1.69.
Schraw, G. (1995). Measures of feeling-of-knowing accuracy: A new look at an old problem. Applied Cognitive Psychology, 9, 321–332. doi:10.1002/acp.2350090405.
Schraw, G., & Roedel, T. D. (1994). Test difficulty and judgment bias. Memory & Cognition, 22(1), 63–69.
Schunn, C. D., Lovett, M. C., & Reder, L. (2001). Awareness and working memory in strategy adaptivity. Memory & Cognition, 29(2), 254–266.
Son, L. K., & Schwartz, B. L. (2002). The relation between metacognitive monitoring and control. In T. J. Perfect, & B. L. Schwartz (Eds.), Applied metacognition (pp. 15–38). Cambridge: Cambridge University Press.
Stahl, E., Pieschl, S., & Bromme, R. (2006). Task complexity, epistemological beliefs and metacognitive calibration: An exploratory study. Journal of Educational Computing Research, 35(4), 319–338. doi:10.2190/1266-0413-387K-7J51.
Sternberg, R. J. (1998). Metacognition, abilities, and developing expertise: What makes an expert student. Instructional Science, 26, 127–140. doi:10.1023/A:1003096215103.
Stone, N. J. (2000). Exploring the relationship between calibration and self-regulated learning. Educational Psychology Review, 12(4), 437–475. doi:10.1023/A:1009084430926.
Veenman, M. V. J. (2003). The assessment of metacognitive skills: What can be learned from multi-method designs? Paper presented at the Biannual Conference of the European Association of Research on Learning and Instruction (EARLI), Padua, Italy.
Veenman, M. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 1(1), 3–14. doi:10.1007/s11409-006-6893-0.
Walczyk, J. J., & Hall, V. C. (1989). Is the failure to monitor comprehension an instance of cognitive impulsivity. Journal of Educational Psychology, 81(3), 294–298. doi:10.1037/0022-0663.81.3.294.
Weingardt, K. R., Leonesio, R. J., & Loftus, E. F. (1994). Viewing eyewitness research from a metacognitive perspective. In J. Metcalfe, & A. P. Shimamura (Eds.), Metacognition, knowing about knowing (pp. 157–184). Cambridge: MIT.
Wiley, J., Griffin, T. D., & Thiede, K. W. (2005). Putting the comprehension in metacomprehension. The Journal of General Psychology, 132(4), 408–428. doi:10.3200/GENP.132.4.408-428.
Winne, P. H. (1995). Inherent details in self-regulated learning. Educational Psychologist, 30(4), 173–187. doi:10.1207/s15326985ep3004_2.
Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8(4), 327–353. doi:10.1016/S1041-6080(96)90022-9.
Winne, P. H. (1997). Experimenting to bootstrap self-regulated learning. Journal of Educational Psychology, 89(3), 397–410. doi:10.1037/0022-0663.89.3.397.
Winne, P. H. (2005). Key issues in modeling and applying research on self-regulated learning. Applied Psychology: An International Review, 54(2), 232–238. doi:10.1111/j.1464-0597.2005.00206.x.
Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41(1), 5–17. doi:10.1207/s15326985ep4101_3.
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Mahwah, NJ: Lawrence Erlbaum Associates.
Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572. doi:10.1016/S0361-476X(02)00006-1.
Winne, P. H., & Jamieson-Noel, D. (2003). Self-regulating studying by objectives for learning: Students’ reports compared to a model. Contemporary Educational Psychology, 28, 259–276. doi:10.1016/S0361-476X(02)00041-3.
Winne, P. H., & Muis, K. R. (2004). Statistical estimates of learner’s judgments about knowledge in calibration of achievement. Unpublished manuscript, Simon Fraser University, Burnaby, BC, Canada.
Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-Regulation (pp. 531–566). San Diego: Academic.
Winograd, P., & Johnston, P. (1982). Comprehension monitoring and the error detection paradigm. Journal of Reading Behavior, XIV(1), 61–76.
Zabrucky, K., & Moore, D. (1989). Children’s ability to use three standards to evaluate their comprehension of text. Reading Research Quarterly, 24, 336–352. doi:10.2307/747773.
Zhao, Q., & Linderholm, T. (2008). Adult metacomprehension: Judgment processes and accuracy constraints. Educational Psychology Review, 20, 191–206. doi:10.1007/s10648-008-9073-8.
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. doi:10.1037/0022-0663.81.3.329.
Acknowledgements
This paper was supported by a grant from the German Research Foundation (DFG) and is based on a chapter of my dissertation which I also presented at AERA 2008, New York. I especially thank the audience of that conference and Prof. Rainer Bromme for helpful comments on earlier versions of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pieschl, S. Metacognitive calibration—an extended conceptualization and potential applications. Metacognition Learning 4, 3–31 (2009). https://doi.org/10.1007/s11409-008-9030-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11409-008-9030-4