Skip to main content

Advertisement

SpringerLink
Log in

Measuring metacognitive skills for mathematics: students’ self-reports versus on-line assessment methods

  • Original Article
  • Published:
ZDM Aims and scope Submit manuscript

Abstract

Various instruments for assessing metacognitive skills and strategy use exist. Off-line self-reports are questionnaires and interviews administered either before or after task performance, while on-line measures are gathered during task performance through thinking aloud or observation. Multi-method studies in reading have shown that off-line methods suffer from serious validity problems, whereas the validity of on-line methods is adequate. Little is known, however, about the validity of methods for assessing metacognition in mathematics. Five instruments were administered to 30 secondary-school students: two prospective questionnaires (MSLQ and ILS) before a mathematics task, two on-line methods (observation and thinking aloud) concurrent to the mathematics task, and a task-specific retrospective questionnaire after the mathematics task. Mathematics performance was assessed by a posttest and GPA. Results confirm that prospective questionnaires have poor convergent and predictive validity in mathematics. Although the retrospective questionnaire does slightly better than prospective questionnaires, the validity of both on-line methods stands out. It is concluded that on-line instruments should be preferred over off-line instruments for the assessment of metacognitive skillfulness in mathematics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aydin, U., & Ubuz, B. (2010). Structural model of metacognition and knowledge of geometry. Learning and Individual Differences, 20, 436–445.

    Article  Google Scholar 

  • Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students’ learning with hypermedia? Journal of Educational Psychology, 96, 523–535.

    Article  Google Scholar 

  • Azevedo, R., Greene, J. A., & Moos, D. C. (2007). The effect of a human agent’s external regulation upon college students’ hypermedia learning. Metacognition and Learning, 2, 67–87.

    Article  Google Scholar 

  • Bannert, M., & Mengelkamp, C. (2008). Assessment of metacognitive skills by means of instruction to think aloud and reflect when prompted. Does the verbalization method affect learning? Metacognition and Learning, 3, 39–58.

    Article  Google Scholar 

  • Brown, A. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation and understanding (pp. 65–116). Hillsdale: Erlbaum.

    Google Scholar 

  • Cromley, J. G., & Azevedo, R. (2006). Self-report of reading comprehension strategies: What are we measuring? Metacognition and Learning, 1, 229–247.

    Article  Google Scholar 

  • De Groot, A. D. (1969). Methodology, foundations of inference and research in the behavioral sciences. The Hague: Mouton.

    Google Scholar 

  • Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28, 425–474.

    Article  Google Scholar 

  • Desoete, A. (2008). Multi-method assessments of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3, 189–206.

    Article  Google Scholar 

  • Desoete, A., & Veenman, M. V. J. (2006). Introduction. In A. Desoete & M. V. J. Veenman (Eds.), Metacognition in mathematics education (pp. 1–10). Hauppauge: Nova Science Publishers.

    Google Scholar 

  • Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3, 231–264.

    Article  Google Scholar 

  • Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20, 391–409.

    Article  Google Scholar 

  • Dunlosky, J., & Nelson, T. O. (1992). Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20, 374–380.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge: MIT Press.

    Google Scholar 

  • Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–235). Hillsdale: Erlbaum.

    Google Scholar 

  • Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911.

    Article  Google Scholar 

  • Gascoine, L., Higgins, S., & Wall, K. (2017). The assessment of metacognition in children aged 4–16 years: a systematic review. Review of Education, 5, 3–57.

    Article  Google Scholar 

  • Guilford, J. P. (1965). Fundamental statistics in psychology and education. New York: McGraw-Hill.

    Google Scholar 

  • Jacobse, A. E., & Harskamp, E. G. (2012). Towards efficient measurement of metacognition in mathematical problem solving. Metacognition and Learning, 7, 133–149.

    Article  Google Scholar 

  • Kramarski, B., & Mevarech, Z. R. (2003). Enhancing mathematical reasoning in the classroom: The effects of cooperative learning and metacognitive training. American Educational Research Journal, 40, 281–310.

    Article  Google Scholar 

  • Li, J., Zhang, B., Du, H., Zhu, Z., & Li, Y. M. (2015). Metacognitive planning: Development and validation of an online measure. Psychological Assessment, 27, 260–271.

    Article  Google Scholar 

  • Meijer, J., Veenman, M. V. J., & van Hout-Wolters, B. H. A. M. (2006). Metacognitive activities in text-studying and problem-solving: Development of a taxonomy. Educational Research and Evaluation, 12, 209–237.

    Article  Google Scholar 

  • Mevarech, Z., & Fridkin, S. (2006). The effects of IMPROVE on mathematical knowledge, mathematical reasoning and meta-cognition. Metacognition and Learning, 1, 85–97.

    Article  Google Scholar 

  • Muis, K. R., Winne, P. H., & Jamieson-Noel, D. (2007). Using a multitrait-multimethod analysis to examine conceptual similarities of three self-regulated learning inventories. British Journal of Educational Psychology, 77, 177–195.

    Article  Google Scholar 

  • Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd edn.). New York: McGraw-Hill.

    Google Scholar 

  • Pape, S. J., & Wang, C. (2003). Middle school children’s strategic behavior: Classification and relation to academic achievement and mathematical problem solving. Instructional Science, 31, 419–449.

    Article  Google Scholar 

  • Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated leaning components of classroom academic performance. Journal of Educational Psychology, 82, 33–40.

    Article  Google Scholar 

  • Pressley, M., & Gaskins, I. (2006). Metacognitive competent reading is constructively responsive reading: How can such reading be developed in students? Metacognition and Learning, 1, 99–113.

    Article  Google Scholar 

  • Schellings, G. (2011). Applying learning strategy questionnaires: Problems and possibilities. Metacognition and Learning, 6, 91–109.

    Article  Google Scholar 

  • Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460–475.

    Article  Google Scholar 

  • Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7, 351–371.

    Article  Google Scholar 

  • Sperling, R. A., Ramsay, C. M., Richmond, A. S., & Klapp, M. (2012). The measurement and predictive ability of metacognition in middle school learners. Journal of Educational Research, 105, 1–7.

    Article  Google Scholar 

  • Van der Stel, M., & Veenman, M. V. J. (2014). Metacognitive skills and intellectual ability of young adolescents: A longitudinal study from a developmental perspective. European Journal of Psychology of Education, 29, 117–137.

    Article  Google Scholar 

  • Van der Stel, M., Veenman, M. V. J., Deelen, K., & Haenen, J. (2010). Development of metacognitive skills in mathematics. ZDM International Journal on Mathematics Education, 42, 219–229.

    Article  Google Scholar 

  • Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In C. Artelt & B. Moschner (Eds.), Lernstrategien und Metakognition: Implikationen für Forschung und Praxis (pp. 75–97). Berlin: Waxmann.

    Google Scholar 

  • Veenman, M. V. J. (2006). The role of intellectual and metacognitive skills in math problem solving. In A. Desoete & M. V. J. Veenman (Eds.), Metacognition in mathematics education (pp. 35–50). Hauppauge: Nova Science Publishers.

    Google Scholar 

  • Veenman, M. V. J. (2007). The assessment and instruction of self-regulation in computer-based environments: A discussion. Metacognition and Learning, 2, 177–183.

    Article  Google Scholar 

  • Veenman, M. V. J. (2008). Giftedness: Predicting the speed of expertise acquisition by intellectual ability and metacognitive skillfulness of novices. In M. F. Shaughnessy, M. V. J. Veenman & C. Kleyn-Kennedy (Eds.), Meta-cognition: A recent review of research, theory, and perspectives (pp. 207–220). Hauppage: Nova Science Publishers.

    Google Scholar 

  • Veenman, M. V. J. (2011). Alternative assessment of strategy use with self-report instruments: A discussion. Metacognition and Learning, 6, 205–211.

    Article  Google Scholar 

  • Veenman, M. V. J. (2013). Training metacognitive skills in students with availability and production deficiencies. In H. Bembenutty, T. Cleary & A. Kitsantas (Eds.), Applications of self-regulated learning across diverse disciplines: A tribute to Barry J. Zimmerman (pp. 299–324). Charlotte: Information Age Publishing.

    Google Scholar 

  • Veenman, M. V. J. (2013). Assessing metacognitive skills in computerized learning environments. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 157–168). New York: Springer.

    Chapter  Google Scholar 

  • Veenman, M. V. J. (2015). Metacognition: ‘Know thyself’. Use that knowledge especially to regulate your own behavior. De Psycholoog, 50, 8–18 (special ed.).

    Google Scholar 

  • Veenman, M. V. J. (2017). Learning to self-monitor and self-regulate. In R. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction (2nd ed., pp. 233–257). New York: Routledge.

    Google Scholar 

  • Veenman, M. V. J., Bavelaar, L., De Wolf, L., & Van Haaren, M. G. P. (2014). The on-line assessment of metacognitive skills in a computerized environment. Learning and Individual Differences, 29, 123–130.

    Article  Google Scholar 

  • Veenman, M. V. J., Elshout, J. J., & Busato, V. V. (1994). Metacognitive mediation in learning with computer-based simulations. Computers in Human Behavior, 10, 93–106.

    Article  Google Scholar 

  • Veenman, M. V. J., Elshout, J. J., & Groen, M. G. M. (1993). Thinking aloud: Does it affect regulatory processes in learning. Tijdschrift voor Onderwijsresearch, 18, 322–330.

    Google Scholar 

  • Veenman, M. V. J., Kerseboom, L., & Imthorn, C. (2000). Test anxiety and metacognitive skillfulness: Availability versus production deficiencies. Anxiety, Stress, and Coping, 13, 391–412.

    Article  Google Scholar 

  • Veenman, M. V. J., Kok, R., & Blöte, A. W. (2005). The relation between intellectual and metacognitive skills at the onset of metacognitive skill development. Instructional Science, 33, 193–211.

    Article  Google Scholar 

  • Veenman, M. V. J., Prins, F. J., & Verheij, J. (2003). Learning styles: Self-reports versus thinking-aloud measures. British Journal of Educational Psychology, 73, 357–372.

    Article  Google Scholar 

  • Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14.

    Article  Google Scholar 

  • Veenman, M. V. J., Wilhelm, P., & Beishuizen, J. J. (2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14, 89–109.

    Article  Google Scholar 

  • Vermunt, J. D. H. M. (1998). The regulation of constructive learning processes. British Journal of Educational Psychology, 68, 149–172.

    Article  Google Scholar 

  • Vuijk, R. A. J., Reichard, L. A., Rozemond, S., Dijkhuis, J. H., Admiraal, C. J., Aalmoes, H., et al. (2003). Getal en Ruimte [Number and Space]. Houten: EPN.

    Google Scholar 

  • Wang, M. C., Haertel, G. D., & Walberg, H. J. (1990). What influences learning? A content analysis of review literature. Journal of Educational Research, 84, 30–43.

    Article  Google Scholar 

  • Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8, 327–353.

    Article  Google Scholar 

  • Winne, P. H. (2014). Issues in researching self-regulated learning as patterns of events. Metacognition and Learning, 9, 229–237.

    Article  Google Scholar 

  • Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibrations of self reports about study tactics and achievement. Contemporary Educational Psychology, 27, 551–572.

    Article  Google Scholar 

  • Zimmerman, B. J., & Martinez-Pons, M. (1990). Student differences in self-regulated learning: Relating grade, sex, and giftedness to self-efficacy and strategy use. Journal of Educational Psychology, 82, 51–59.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Metacognition Research, Mesdaglaan 50, 2182SX, Hillegom, The Netherlands

    Marcel V. J. Veenman & Dorit van Cleef

Authors
  1. Marcel V. J. Veenman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dorit van Cleef
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcel V. J. Veenman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Veenman, M.V.J., van Cleef, D. Measuring metacognitive skills for mathematics: students’ self-reports versus on-line assessment methods. ZDM Mathematics Education 51, 691–701 (2019). https://doi.org/10.1007/s11858-018-1006-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11858-018-1006-5

Keywords

Search

Navigation