Skip to main content
SpringerLink
Log in

Exploring General Versus Task-Specific Assessments of Metacognition in University Chemistry Students: A Multitrait–Multimethod Analysis

  • Published:
Research in Science Education Aims and scope Submit manuscript

Abstract

The purpose of this study was to use multiple assessments to investigate the general versus task-specific characteristics of metacognition in dissimilar chemistry topics. This mixed-method approach investigated the nature of undergraduate general chemistry students’ metacognition using four assessments: a self-report questionnaire, assessment of concurrent metacognitive skills, confidence judgment, and calibration accuracy. Data were analyzed using a multitrait–multimethod correlation matrix, supplemented with regression analyses, and qualitative interpretation. Significant correlations among task performance, calibration accuracy, and concurrent metacognition within a task suggest a converging relationship. Confidence judgment, however, was not associated with task performance or the other metacognitive measurements. The results partially support hypotheses of both general and task-specific metacognition. However, general and task-specific properties of metacognition were detected using different assessments. Case studies were constructed for two participants to illustrate how concurrent metacognition varied within different task demands. Considerations of how each assessment may appropriate different metacognitive constructs and the importance of the alignment of analytical constructs when using multiple assessments are discussed. These results may help lead to improvements in metacognition assessment and may provide insights into designs of effective metacognitive instruction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209.

    Article  Google Scholar 

  • Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54(2), 199–231.

    Article  Google Scholar 

  • Bonett, D. G., & Wright, T. A. (2000). Sample size requirements for estimating Pearson, Kendall and Spearman correlations. Psychometrika, 65(1), 23–28.

    Article  Google Scholar 

  • Burkett, C., & Azevedo, R. (2012). The effect of multimedia discrepancies on metacognitive judgments. Computers in Human Behavior, 28(4), 1276–1285.

    Article  Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56(2), 81–105.

    Article  Google Scholar 

  • Chiu, J., & Linn, M. (2012). The role of self-monitoring in learning chemistry with dynamic visualizations. In A. Zohar & Y. J. Dori (Eds.), Metacognition in science education: Trends in current research (pp. 133–163). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Clarebout, G., Elen, J., & Onghena, P. (2006). In search of the reliability of a Flemish version of the knowledge monitoring assessment test. Metacognition and Learning, 1(2), 137–147.

    Article  Google Scholar 

  • Cleary, T. J. (2011). Emergence of self-regulated learning microanalysis: Historical overview, essential features, and implications for research and practice. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 329–345). New York: Routledge.

    Google Scholar 

  • Conner, L. (2007). Cueing metacognition to improve researching and essay writing in a final year high school biology class. Research in Science Education, 37(1), 1–16.

    Article  Google Scholar 

  • Cooper, M. M., & Sandi-Urena, S. (2009). Design and validation of an instrument to assess metacognitive skillfulness in chemistry problem solving. Journal of Chemical Education, 86(2), 240–245.

    Article  Google Scholar 

  • Cooper, M. M., Sandi-Urenab, S., & Stevens, R. (2008). Reliable multi method assessment of metacognition use in chemistry problem solving. Chemistry Education Research and Practice, 9, 18–24.

    Article  Google Scholar 

  • Coutinho, S., Wiemer-Hastings, K., Skowronski, J. J., & Britt, M. A. (2005). Metacognition, need for cognition and use of explanations during ongoing learning and problem solving. Learning and Individual Differences, 15(4), 321–337.

  • Cromley, J. G., Snyder-Hogan, L. E., & Luciw-Dubas, U. A. (2010). Cognitive activities in complex science text and diagrams. Contemporary Educational Psychology, 35(1), 59–74.

    Article  Google Scholar 

  • David, F. N. (1938). Tables of the ordinates and probability integral of the distribution of the correlation coefficient in small samples. New York: Cambridge University Press.

    Google Scholar 

  • Desoete, A., Roeyers, H., & De Clercq, A. (2003). Can offline metacognition enhance mathematical problem solving? Journal of Educational Psychology, 95(1), 188–200.

    Article  Google Scholar 

  • Dinsmore, D. L., & Parkinson, M. M. (2013). What are confidence judgments made of? Students’ explanations for their confidence ratings and what that means for calibration. Learning and Instruction, 24, 4–14.

    Article  Google Scholar 

  • Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction, 22(4), 271–280.

    Article  Google Scholar 

  • Dunlosky, J., & Thiede, K. W. (2013). Four cornerstones of calibration research: Why understanding students’ judgments can improve their achievement. Learning and Instruction, 24, 58–61.

    Article  Google Scholar 

  • Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational Research Review, 1(1), 3–14.

    Article  Google Scholar 

  • Ertmer, P. A., & Newby, T. J. (1996). The expert learner: Strategic, self-regulated, and reflective. Instructional Science, 24(1), 1–24.

    Article  Google Scholar 

  • Fisher, R. A. (1932). Statistical methods for research workers. London: Oliver & Boyd.

    Google Scholar 

  • Flavell, J. H., Miller, P. H., & Miller, S. A. (2002). Cognitive development (4th ed.). Upper Saddle River: Prentice Hall.

    Google Scholar 

  • Ford, C. L., & Yore, L. D. (2012). Toward convergence of critical thinking, metacognition, and reflection: Illustrations from natural and social sciences, teacher education, and classroom practice. In A. Zohar & Y. J. Dori (Eds.), Metacognition in science education: Trends in current research (pp. 251–271). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Furió, C., Calatayud, M. L., Bárcenas, S. L., & Padilla, O. M. (2000). Functional fixedness and functional reduction as common sense reasonings in chemical equilibrium and in geometry and polarity of molecules. Science Education, 84(5), 545–565.

    Article  Google Scholar 

  • Georghiades, P. (2006). The role of metacognitive activities in the contextual use of primary pupils’ conceptions of science. Research in Science Education, 36(1–2), 29–49.

    Article  Google Scholar 

  • Greene, J. A., & Azevedo, R. (2009). A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemporary Educational Psychology, 34(1), 18–29.

    Article  Google Scholar 

  • Hong, E., & Peng, Y. (2008). Do Chinese students’ perceptions of test value affect test performance? Mediating role of motivational and metacognitive regulation in test preparation. Learning and Instruction, 18(6), 499–512.

    Article  Google Scholar 

  • Howard, B. C., McGee, S., Shia, R., & Hong, N. S. (2000). Metacognitive self-regulation and problem-solving: Expanding the theory base through factor analysis. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Retrieved from ERIC database. (ED470973)

  • Kelemen, W. L., Frost, P. J., & Weaver, C. A. (2000). Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition, 28(1), 92–107.

    Article  Google Scholar 

  • Koch, A. (2001). Training in metacognition and comprehension of physics texts. Science Education, 85(6), 758–768.

    Article  Google Scholar 

  • Koriat, A., & Levy-Sardot, R. (1999). Processes underlying metacognitive judgments: Information-based and experience-based monitoring of one’s own knowledge. In S. Chaiken & Y. Trope (Eds.), Dual-process theories in social psychology (pp. 483–502). New York: Guilford.

    Google Scholar 

  • Kramarski, B., Mevarech, Z. R., & Lieberman, A. (2001). Effects of multilevel versus unilevel metacognitive training on mathematical reasoning. Journal of Educational Research, 94(5), 292–300.

    Article  Google Scholar 

  • Leou, M., Abder, P., Riordan, M., & Zoller, U. (2006). Using ‘HOCS-centered learning’ as a pathway to promote science teachers' metacognitive development. Research in Science Education, 36(1–2), 69–84.

    Article  Google Scholar 

  • Loverude, M. E., Kautz, C. H., & Heron, P. R. L. (2002). Student understanding of the first law of thermodynamics: Relating work to the adiabatic compression of an ideal gas. American Journal of Physics, 70(2), 137–148.

    Article  Google Scholar 

  • Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97(4), 723–731.

    Article  Google Scholar 

  • Meltzer, D. E. (2004). Investigation of students’ reasoning regarding heat, work, and the first law of thermodynamics in an introductory calculus-based general physics course. American Journal of Physics, 72(11), 1432–1446.

    Article  Google Scholar 

  • Moos, D. C., & Azevedo, R. (2008). Monitoring, planning, and self-efficacy during learning with hypermedia: The impact of conceptual scaffolds. Computers in Human Behavior, 24(4), 1686–1706.

    Article  Google Scholar 

  • Muis, K. R., Winne, P. H., & Jamieson-Noel, D. (2007). Using a multitrait-multimethod analysis to examine conceptual similarities of three self-regulated learning inventories. British Journal of Educational Psychology, 77(1), 177–195.

    Article  Google Scholar 

  • Nietfeld, J. L., & Schraw, G. (2002). The effect of knowledge and strategy training on monitoring accuracy. Journal of Educational Research, 95(3), 131–142.

    Article  Google Scholar 

  • Nietfeld, J. L., Cao, L., & Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom. Journal of Experimental Education, 74(1), 7–28.

    Google Scholar 

  • Ozuru, Y., Kurby, C. A., & McNamara, D. S. (2012). The effect of metacomprehension judgment task on comprehension monitoring and metacognitive accuracy. Metacognition and Learning, 7(2), 113–131.

    Article  Google Scholar 

  • Peterson, R. F., Treagust, D. F., & Garnett, P. (1989). Development and application of a diagnostic instrument to evaluate grade-11 and -12 students’ conceptions of covalent bonding and structure following a course of instruction. Journal of Research in Science Teaching, 26(4), 301–314.

    Article  Google Scholar 

  • Pieschl, S. (2009). Metacognitive calibration—an extended conceptualization and potential applications. Metacognition and Learning, 4(1), 3–31.

    Article  Google Scholar 

  • Pieschl, S., Stahl, E., Murray, T., & Bromme, R. (2012). Is adaptation to task complexity really beneficial for performance? Learning and Instruction, 22(4), 281–289.

    Article  Google Scholar 

  • Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.

    Article  Google Scholar 

  • Prins, F. J., Veenman, M. V. J., & Elshout, J. J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory. Learning and Instruction, 16(4), 374–387.

    Article  Google Scholar 

  • Reyes, M. L. (2011). Domain specificity of motivation and learning strategies: Filipino science high school students’ profiles in science and social studies. Asia-Pacific Education Researcher, 20(2), 308–321.

    Google Scholar 

  • Rotgans, J., & Schmidt, H. (2009). Examination of the context–specific nature of self–regulated learning. Educational Studies, 35(3), 239–253.

    Article  Google Scholar 

  • Sandi-Urena, S., Cooper, M. M., & Stevens, R. H. (2010). Enhancement of metacognition use and awareness by means of a collaborative intervention. International Journal of Science Education, 33(3), 323–340.

    Article  Google Scholar 

  • Schellings, G. (2011). Applying learning strategy questionnaires: Problems and possibilities. Metacognition and Learning, 6(2), 91–109.

    Article  Google Scholar 

  • Schellings, G., & Broekkamp, H. (2011). Signaling task awareness in think-aloud protocols from students selecting relevant information from text. Metacognition and Learning, 6(1), 65–82.

    Article  Google Scholar 

  • Schneider, W., Schlagmüller, M., & Visé, M. (1998). The impact of metamemory and domain-specific knowledge on memory performance. European Journal of Psychology of Education, 13(1), 91–103.

    Article  Google Scholar 

  • Schraw, G. (1995). Measures of feeling-of-knowing accuracy: A new look at an old problem. Applied Cognitive Psychology, 9(4), 321–332.

    Article  Google Scholar 

  • Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4(1), 33–45.

    Article  Google Scholar 

  • Schraw, G., Dunkle, M. E., Bendixen, L. D., & Roedel, T. D. (1995). Does a general monitoring skill exist? Journal of Educational Psychology, 87(3), 433–444.

    Article  Google Scholar 

  • Schraw, G., Crippen, K. J., & Hartley, K. (2006). Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Research in Science Education, 36(1–2), 111–139.

    Article  Google Scholar 

  • Schwartz, D. L., Bransford, J. D., & Sears, D. (2005). Efficiency and innovation in transfer. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (pp. 1–51). Greenwich: Information Age.

    Google Scholar 

  • Shin, N., Jonassen, D. H., & McGee, S. (2003). Predictors of well-structured and ill-structured problem solving in an astronomy simulation. Journal of Research in Science Teaching, 40(1), 6–33.

    Article  Google Scholar 

  • Son, L. K., & Metcalfe, J. (2005). Judgments of learning: Evidence for a two-stage process. Memory & Cognition, 33(6), 1116–1129.

    Article  Google Scholar 

  • Song, C., Kanai, R., Fleming, S. M., Weil, R. S., Schwarzkopf, D. S., & Rees, G. (2011). Relating inter-individual differences in metacognitive performance on different perceptual tasks. Consciousness & Cognition, 20(4), 1787–1792.

    Article  Google Scholar 

  • Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27(1), 51–79.

    Article  Google Scholar 

  • Taber, K. S. (2002). Chemical misconception—Prevention, diagnosis and cure: vol. I: Theoretical background. London: Royal Society of Chemistry.

    Google Scholar 

  • Thomas, G. P. (2013). Changing the metacognitive orientation of a classroom environment to stimulate metacognitive reflection regarding the nature of physics learning. International Journal of Science Education, 35(7), 1183–1207.

    Article  Google Scholar 

  • Thomas, P. L., & Schwenz, R. W. (1998). College physical chemistry students’ conceptions of equilibrium and fundamental thermodynamics. Journal of Research in Science Teaching, 35(10), 1151–1160.

    Article  Google Scholar 

  • van der Stel, M., & Veenman, M. V. J. (2008). Relation between intellectual ability and metacognitive skillfulness as predictors of learning performance of young students performing tasks in different domains. Learning and Individual Differences, 18(1), 128–134.

    Article  Google Scholar 

  • van der Stel, M., & Veenman, M. V. J. (2010). Development of metacognitive skillfulness: A longitudinal study. Learning and Individual Differences, 20(3), 220–224.

    Article  Google Scholar 

  • van Loon, M. H., de Bruin, A. B. H., van Gog, T., & van Merriënboer, J. J. G. (2013). Activation of inaccurate prior knowledge affects primary-school students’ metacognitive judgments and calibration. Learning and Instruction, 24, 15–25.

    Article  Google Scholar 

  • Veenman, M. V. J. (2012). Metacognition in science education: Definitions, constituents, and their intricate relation with cognition. In A. Zohar & Y. J. Dori (Eds.), Metacognition in science education: Trends in current research (pp. 21–36). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Veenman, M. V. J., & Spaans, M. A. (2005). Relation between intellectual and metacognitive skills: Age and task differences. Learning and Individual Differences, 15(2), 159–176.

    Article  Google Scholar 

  • Veenman, M. V. J., & Verheij, J. (2003). Identifying technical students at risk: Relating general versus specific metacognitive skills to study success. Learning and Individual Differences, 13, 259–272.

    Article  Google Scholar 

  • Veenman, M. V. J., Elshout, J. J., & Meijer, J. (1997). The generality vs. domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187–209.

    Article  Google Scholar 

  • Veenman, M. V. J., van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1(1), 3–14.

    Article  Google Scholar 

  • Vidal-Abarca, E., Mañá, A., & Gil, L. (2010). Individual differences for self-regulating task-oriented reading activities. Journal of Educational Psychology, 102(4), 817–826.

    Article  Google Scholar 

  • Wang, C.-Y. (2007). The role of mental-modeling ability, content knowledge, and mental models in general chemistry students’ understanding about molecular polarity (Doctoral dissertation). Columbia: University of Missouri.

  • Wang, C.-Y. (2013). Scaffolding middle school students’ construction of scientific explanations. Paper presented at the biannual conference of the European Science Education Research Association, Nicosia, Cyprus.

  • Wang, C.-Y., & Barrow, L. H. (2011). Characteristics and levels of sophistication: An analysis of chemistry students’ ability to think with mental models. Research in Science Education, 41(4), 561-586

  • Wang, J.-R., & Chen, S.-F. (2014). Exploring mediating effect of metacognitive awareness on comprehension of science texts through structural equation modeling analysis. Journal of Research in Science Teaching, 51, 175–191.

    Article  Google Scholar 

  • Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572.

    Article  Google Scholar 

  • Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, M. Zeidner, & P. R. Pintrich (Eds.), Handbook of self-regulation (pp. 531–566). San Diego: Academic.

    Chapter  Google Scholar 

  • Winne, P. H., Jamieson-Noel, D., & Muis, K. (2002). Methodological issues and advances in researching tactics, strategies, and self-regulated learning. In P. R. Pintrich & M. L. Maehr (Eds.), Advances in motivation and achievement: New directions in measures and methods (pp. 121–155). Greenwich: JAI Press.

    Google Scholar 

  • Wolters, C. A., & Pintrich, P. R. (1998). Contextual differences in student motivation and self-regulated learning in mathematics, English, and social studies classrooms. Instructional Science, 26, 27–47.

    Article  Google Scholar 

  • Yürük, N., Beeth, M. E., & Andersen, C. (2009). Analyzing the effect of metaconceptual teaching practices on students’ understanding of force and motion concepts. Research in Science Education, 39(4), 449–475.

    Article  Google Scholar 

  • Zohar, A., & Barzilai, S. (2013). A review of research on metacognition in science education: Current and future directions. Studies in Science Education, 49(2), 121–169.

    Article  Google Scholar 

Download references

Acknowledgments

Funding for this research work was supported by the National Science Council, Taiwan, under grant numbers NSC 98-2511-S-009-008 and NSC 99-2511-S-009-002-MY2. The author would also like to thank professor David E. Meltzer for the use of interview questions and the Center of Educational Technology at Wheeling Jesuit University for the use of the Inventory of Metacognitive Self-Regulation.

Author information

Authors and Affiliations

  1. Institute of Education, National Chiao Tung University, 1001 Ta-Hsueh Road, Hsinchu, 300, Taiwan

    Chia-Yu Wang

Authors
  1. Chia-Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chia-Yu Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, CY. Exploring General Versus Task-Specific Assessments of Metacognition in University Chemistry Students: A Multitrait–Multimethod Analysis. Res Sci Educ 45, 555–579 (2015). https://doi.org/10.1007/s11165-014-9436-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11165-014-9436-8

Keywords

Search

Navigation