Skip to main content
SpringerLink
Log in

Assessment of metacognitive skills by means of instruction to think aloud and reflect when prompted. Does the verbalisation method affect learning?

  • Published:
Metacognition and Learning Aims and scope Submit manuscript

Abstract

Recent research on metacognition points out the crucial role of on-line methods when endeavouring to conduct valid assessments of metacognitive skills. Presently, different on-line methods are used, however, it is still a question of research whether and how they affect students’ learning behaviour and learning outcome. Thus, the aim of this study is to quasi-experimentally analyse the effects of two on-line verbalisation methods on learning performance. By means of the thinking-aloud method, students in one experimental group (n = 24) were instructed to read and think aloud during learning. With the reflection when prompted method, students of another experimental group (n = 24) were prompted at each navigational step to reflect on the reasons why they chose specific information. Students in the control group (n = 22) learned without being instructed to verbalise. All three groups were treated identically except for the different use of verbalisation assessment methods. The students’ task was to learn the concepts and principles of operant conditioning presented in a hypermedium within 30 min. The students’ learning sessions were videotaped and learning performance was obtained immediately afterwards. Based on Ericsson and Simon’s (Protocol analysis: Verbal reports as data, MIT, Cambridge, 1993) model, no performance differences between the thinking aloud and the control group were hypothesised. However, prompting students for metacognitive reflection should affect learning performance positively, which is confirmed by the results only in tendency for transfer performance. Implications for on-line assessment methods of metacognitive skill will be discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Afflerbach, P. (2000). Verbal reports and protocol analysis. In M. L. M. Kamil, P. B. Mosenthal, D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 163–179). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (1999). IST 2000—Intelligenz-Struktur-Test 2000 [Intelligence Structure Test 2000]. Göttingen: Hogrefe.

    Google Scholar 

  • Anderson, J. R. (2004). Cognitive psychology and its implications. New York: Worth.

    Google Scholar 

  • Artelt, C. (2000). Strategisches Lernen [Strategic Learning]. Münster: Waxmann.

    Google Scholar 

  • Astleitner, H. (1997). Lernen in Informationsnetzen. Theoretische Aspekte und empirische Analysen des Umgangs mit neuen Informationstechnologien aus erziehungswissenschaftlicher Perspektive [Learning with information networks. Theoretical aspects and empirical analyses of dealing with new information technology from an educational perspective]. Frankfurt: Europäischer Verlag der Wissenschaften.

    Google Scholar 

  • Baker, L. (1994). Fostering metacognitive development. Advances in Child Development and Behavior, 25, 201–239.

    Google Scholar 

  • Balcytiene, A. (1999). Exploring individual processes of knowledge construction with hypertext. Instructional Science, 27, 303–328.

    Google Scholar 

  • Bannert, M. (2003). Effekte metakognitiver Lernhilfen auf den Wissenserwerb in vernetzten Lernumgebungen [Effects of metacognitive help on knowledge acquisition in net-based learning environments]. Zeitschrift für Pädagogische Psychologie, 17(1), 13–25.

    Article  Google Scholar 

  • Bannert, M. (2004). Designing metacognitive support for hypermedia learning. In H. Niegemann, D. Leutner, & R. Brünken (Eds.), Instructional design for multimedia-learning (pp. 19–30). Münster: Waxmann.

    Google Scholar 

  • Bannert, M. (2005). Explorationsstudie zum spontanen metakognitiven Strategie-Einsatz in hypermedialen Lernumgebungen [Exploring the spontaneous use of metacognitive strategies during web-based learning]. In C. Artelt & B. Moschner (Eds.), Lernstrategien und Metakognition: Implikationen für Forschung und Praxis (pp. 127–151). Münster: Waxmann.

    Google Scholar 

  • Bannert, M. (2006). Effects of reflection prompts when learning with hypermedia. Journal of Educational Computing Research, 4, 359–375.

    Article  Google Scholar 

  • Bannert, M. (2007). Metakognition beim Lernen mit Hypermedien. [Metacognition in Hypermedia Learning]. Münster: Waxmann.

    Google Scholar 

  • Beaseley, R. E., & Waugh, M. L. (1995). Cognitive mapping architectures and hypermedia disorientation: An empirical study. Journal of Educational Multimedia and Hypermedia, 4, 239–255.

    Google Scholar 

  • Beishuizen, J. J., & Stoutjesdijk, E. T. (1999). Study strategies in a computer assisted study environment. Learning and Instruction, 9, 281–301.

    Article  Google Scholar 

  • Boekaerts, M. (1997). Self-Regulated Learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and Instruction, 7, 161–186.

    Article  Google Scholar 

  • Brenstein, E., & Schellhas, B. (1998). Lernstrategien in hypermedialen Lernumgebungen [Learning strategies in hypermedia learning environments]. Vortrag auf dem 39. Kolloquium zur Lern- und Lehrforschung. Universität Potsdam.

  • Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructional psychology (pp. 77–165). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6, 271–315.

    Article  Google Scholar 

  • Crain-Thoreson, C., Lippman, M. Z., & McClendon-Magnuson, D. (1997). Windows on comprehension: Reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology, 89(4), 579–591.

    Article  Google Scholar 

  • Crutcher, R. J. (1994). Telling what we know: The use of verbal report methodologies in psychological research. Psychological Science, 5(5), 241–244.

    Article  Google Scholar 

  • Davidson-Shivers, G. V., Rasmussen, K. L., & Bratton-Jeffery, M. F. (1997). Investigating learning strategies generation in a hypermedia environment using qualitative methods. Journal of Computing in Childhood Education, 8(2/3), 247–261.

    Google Scholar 

  • Davidson-Shivers, G. V., Shorter, L., & Jordan, K., & Rasmussen, K. L. (1999). Learning strategies and navigation decisions of children using a hypermedia lesson. Journal of Educational Multimedia and Hypermedia, 8(2), 175–188.

    Google Scholar 

  • De Jong, F. (1987). Differences in the self-regulation processes between successful and less successful students and the prediction of learning performances in case of comprehension and learning of text. In P. R. J. Simons & G. Beukhof (Eds.), Regulation of learning (pp. 33–45). Den Haag: SVO-Selecta.

    Google Scholar 

  • Desoete, A., Roeyers, H., & De Clercq, A. (2003). Can off-line metacognition enhance mathematical problem solving? Journal of Educational Psychology, 95(1), 188–200.

    Article  Google Scholar 

  • Dominowski, R. L. (1998). Verbalization and problem solving. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 25–45). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Drewniak, U. (1992). Lernen mit Bildern in Texten. Untersuchung zur Optimierung des Lernerfolgs bei Benutzung computerpräsentierter Texte und Bilder [Learning with pictures in texts. Investigation to optimise learning success by using computer presented texts and pictures]. Münster: Waxmann.

    Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge: MIT.

    Google Scholar 

  • Flavell, J. H., & Wellman, H. M. (1977). Metamemory. In R. V. Kail & W. Hagen (Eds.), Perspectives on development of memory and cognition (pp. 3–31). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Foltz, P. W. (1996). Comprehension, coherence, and strategies in hypertext and linear text. In J. F. Rouet, J. J. Levonen, A. Dillon, & R. J. Spiro (Eds.), Hypertext and cognition (pp. 109–136). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Garner, R. (1988). Verbal-report data on cognitive and metacognitive strategies. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies. Issues in assessment, instruction, and evaluation (pp. 63–76). San Diego, CA: Academic.

    Google Scholar 

  • Hasselhorn, M. (1992). Metakognition und Lernen [Metacognition and learning]. In G. Nold (Ed.), Lernbedingungen und Lernstrategien. Welche Rolle spielen kognitive Verstehensstrukturen? (pp. 35–63). Tübingen: Gunter Narr Verlag.

    Google Scholar 

  • Hasselhorn, M. (1995). Kognitives Training: Grundlagen, Begrifflichkeiten und Desiderate [Cognitive training: Foundations, terms, and desiderates]. In W. Hager (Ed.), Programme zur Förderung des Denkens bei Kindern (pp. 14–40). Göttingen: Hogrefe.

    Google Scholar 

  • Hermans, H., Petermann, F., & Zielinski, W. (1978). LMT—Leistungsmotivationstest [LMT—performance motivation test]. Amsterdam: Swets & Zeitlinger.

    Google Scholar 

  • Hill, J. R., & Hannafin, M. J. (1997). Cognitive strategies and learning from the World Wide Web. Educational Technology Research and Development, 45(4), 37–64.

    Article  Google Scholar 

  • Jacobson, M. J., Maouri, C., Mishra, P., & Kolar, C. (1996). Learning with hypertext learning environments: Theory, design and research. Journal of Educational Multimedia and Hypermedia, 5(3/4), 239–281.

    Google Scholar 

  • Kramarski, B., & Feldman, Y. (2000). Internet in the classroom: Effects on reading comprehension, motivation and metacognitive awareness. Educational Media International, 37(3), 149–155.

    Article  Google Scholar 

  • Lawless, K. A., & Brown, S. W. (1997). Multimedia learning environments: Issues of learner control and navigation. Instructional Science, 25, 117–131.

    Article  Google Scholar 

  • Lin, X., & Lehman, J. D. (1999). Supporting learning of variable control in a computer-based biology environment: Effects of prompting college students to reflect on their own thinking. Journal of Research in Science Teaching, 36(7), 837–858.

    Article  Google Scholar 

  • Lin, X., Hmelo, C., Kinzer, C. K., & Secules, T. J.(1999). Designing technology to support reflection. Educational Technology Research and Development, 47(3), 43–62.

    Article  Google Scholar 

  • Lompscher, J. (1996). Erfassung von Lernstrategien auf der Reflexionsebene [Assessment of learning strategies on reflection level]. Empirische Pädagogik. Zeitschrift zu Theorie und Praxis erziehungswissenschaftlicher Forschung, 10(3), 245–276.

    Google Scholar 

  • Low, B. R., & Sweller, J. (2005). The modality principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 147–158). Cambridge: Cambridge University Press.

    Google Scholar 

  • McManus, T. F. (2000). Individualizing instruction in a web-based hypermedia learning environment: Nonlinearity, advanced organizers, and self regulated learners. Journal of Interactive Learning Research, 11, 219–251.

    Google Scholar 

  • Mengelkamp, C. (2001). Lautes Denken zur Erfassung von Lernstrategien [Thinking aloud to assess learning strategies]. Unveröffentlichte Diplomarbeit, Fachbereich Psychologie, Universität Koblenz-Landau.

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.

    Article  Google Scholar 

  • Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.

    Article  Google Scholar 

  • Payne, J. W. (1994). Thinking aloud: Insights into information processing. Psychological Science, 5(5), 241–248.

    Article  Google Scholar 

  • Pintrich, P. R., Wolters, C. A., & Baxter, G. P. (2000). Assessing metacognition and self-regulated learning. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 43–97). Lincoln, NE: Buros Institute of Mental Measurements.

    Google Scholar 

  • Pressley, M. (2000). Development of grounded theories of complex cognitive processing: Exhaustive within- and between-study analyses of think-aloud data. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 262–296). Lincoln, NB: Buros Institute of Mental Measurements, University of Nebraska-Lincoln.

    Google Scholar 

  • Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Pressley, M., Borkowski, J. G., & Schneider, W. (1987). Cognitive strategies: Good strategy users coordinate metacognition and knowledge. In R. Vasta & G. Whitehurst (Eds.), Annals of child development (Vol. 4, pp. 89–129). Greenwich, CT: JAI.

    Google Scholar 

  • Pressley, M., Borkowski, J. G., & Schneider, W. (1989). Good information processing: What it is and how education can do to promote it. International Journal of Educational Research, 13, 857–867.

    Article  Google Scholar 

  • Prins, F. J. (2002). Search & see. The roles of metacognitve skillfulness and intellectual ability during novice inductive learning in a complex computer-simulated environment. Enschede, Niederlande: PrintPartners Ipskamp.

    Google Scholar 

  • L. M. Reder (Ed.) (1996). Implicit memory and metacognition. Mahwah, NJ: Erlbaum.

  • Rouet, J. F., & Levonen, J. J. (1996). Studying and learning with hypertext: Empirical studies and their implications. In J. F. Rouet, J. J. Levonen, A. Dillon, & R. J. Spiro (Eds.), Hypertext and cognition (pp. 9–23). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Russo, J., Johnson, E., & Stephens, D. (1989). The validity of verbal protocols. Memory and Cognition, 17, 759–769.

    Google Scholar 

  • Schneider, W. (1996). Zum Zusammenhang zwischen Metakognition und Motivation bei Lern- und Gedächtnisvorgängen [Coherence between metacognition and motivation in learning and memory processes]. In C. Spiel, U. Kastner-Koller, & P. Deimann (Eds.), Motivation und Lernen aus der Perspektive lebenslanger Entwicklung (pp. 121–133). Münster: Waxmann.

    Google Scholar 

  • W. Schneider & F. E. Weinert (Eds.) (1990). Interactions among aptitudes, strategies, and knowledge in cognitive performance. New York: Springer.

  • Schnotz, W. (1992). Metacognition and self-regulation in text processing: Some comments. In M. Carretero, M. L. Pope, R. J. Simons, & J. I. Pozo (Eds.), Learning & instruction. European research in an international context (Vol. 3, pp. 365–375). Oxford: Pergamon.

    Google Scholar 

  • Schnotz, W. (1998). Strategy-specific information access in knowledge acquisition from hypertext. In L. B. Resnick, R. Säljö, C. Pontecorvo, & B. Burge (Eds.), Discourse, tools, and reasoning. Essays on situated cognition. Heidelberg: Springer.

    Google Scholar 

  • Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156.

    Article  Google Scholar 

  • Schooler, J. W., Ohlsson, S., & Brooks, K. (1993). Thoughts beyond words: When language overshadows insight. Journal of Experimental Psychology: General, 122, 166–183.

    Article  Google Scholar 

  • Schraw, G. (2001). Promoting general metacognitive awareness. In H. Hartman (Ed.), Metacognition in learning and instruction. Theory, research and practice (pp. 3–16). Dordrecht: Kluwer.

    Google Scholar 

  • G. Schraw & J. C. Impara (Eds.) (2000). Issues in the measurement of metacognition. Lincoln, NE: Buros Institute of Mental Measurements.

  • Schreiber, B. (1998). Selbstreguliertes Lernen. Entwicklung und Evaluation von Trainingsansätzen für Berufstätige [Self-regulated learning. Development and evaluation of training approaches for the employed]. Münster: Waxmann.

    Google Scholar 

  • Simons, P. R. J., & De Jong, F. P. (1992). Self-regulation and computer-assisted instruction. Applied Psychology: An International Review, 41, 333–346.

    Article  Google Scholar 

  • van den Boom, G., Paas, F., van Merrienboer, J. J. G., & van Gog, T. (2004). Reflection prompts and tutor feedback in a web-based learning environment: Effects on students’ self-regulated learning competence. Computers in Human Behavior, 20, 551–567.

    Article  Google Scholar 

  • van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method. London: Academic.

    Google Scholar 

  • Veenman, M. V. J. (1993). Intellectual ability and metacognitive skill: Determinants of discovery learning in computerized learning environments. Amsterdam: University of Amsterdam.

    Google Scholar 

  • Veenman, M. V. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In C. Artelt & B. Moschner (Eds.), Lernstrategien und Metakognition: Implikationen für Forschung und Praxis (pp. S. 77–99). Berlin: Waxmann.

    Google Scholar 

  • Weinand, C. (2003). Metakognitive Förderung beim hypermedialem Lernen [Metacognitive support on hypermedia learning]. Unveröffentlichte Diplomarbeit, Fachbereich Psychologie, Universität Koblenz-Landau.

  • Wild, K. P., Schiefele, U., & Winteler, A. (1992). LIST. Ein Verfahren zur Erfassung von Lernstrategien im Studium [LIST. A scale for assessing learning strategies at studies]. (Gelbe Reihe: Arbeiten zur Empirischen Pädagogik und Pädagogischen Psychologie, Nr. 20). Neubiberg: Universität der Bundeswehr, Institut für Erziehungswissenschaft und Pädagogische Psychologie.

    Google Scholar 

  • Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). San Diego, CA: Academic.

    Google Scholar 

Download references

Acknowledgement

This study was partially supported by the DFG [Deutsche Forschungsgemeinschaft, BA2044/1].

Author information

Authors and Affiliations

  1. Faculty of Philosophy, Educational Media, Chemnitz University of Technology, Reichenhainertsr. 41, 09126, Chemnitz, Germany

    Maria Bannert

  2. Center of Educational Research, University of Koblenz–Landau, Landau, Germany

    Christoph Mengelkamp

Authors
  1. Maria Bannert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Mengelkamp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Bannert.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bannert, M., Mengelkamp, C. Assessment of metacognitive skills by means of instruction to think aloud and reflect when prompted. Does the verbalisation method affect learning?. Metacognition Learning 3, 39–58 (2008). https://doi.org/10.1007/s11409-007-9009-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11409-007-9009-6

Keywords

Search

Navigation