Proficient readers engage in a wide range of cognitive and metacognitive strategies, and both developmental and classroom intervention researchers are in need of high-quality measures of strategy use. Several researchers have recently called into question the validity of the most common type of measures of strategy use in reading, self-report or introspective measures (i.e., the participant must report on his or her cognitive activity while not actually engaged in the activity). We administered three parallel strategy use measures to a sample of 30 ninth-grade students: a prospective self-report measure, a concurrent multiple-choice measure which required students to apply the strategies to specific passages, and a text on which we asked students to think aloud. We also collected two measures of reading comprehension—a standardized measure and free recall scores. Consistent with Veenman’s (2005) conclusions based on a literature review, the concurrent multiple-choice and think-aloud data were both significantly correlated with both of the comprehension scores and with each other, whereas the prospective self-report data had non-significant correlations with all of the other measures. We conclude by recommending concurrent measures for researchers who wish to study strategy use in reading comprehension.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Afflerbach, P. (2000). Verbal reports and protocol analysis. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research, vol. 3 (pp. 163–179). Mahwah, NJ: Erlbaum.
Alexander, P. A., & Judy, J. E. (1988). The interaction of domain-specific and strategic knowledge in academic performance. Review of Educational Research, 58(4), 375–404.
Alexander, P. A., Murphy, P. K., Woods, B. S., Duhon, K. E., & Parker, D. (1997). College instruction and concomitant changes in students’ knowledge, interest, and strategy use: A study of domain learning. Contemporary Educational Psychology, 22(2), 125–146.
Alfassi, M. (2004). Reading to learn: Effects of combined strategy instruction on high school students. Journal of Educational Research, 97(4), 171–184.
Artelt, C., Schiefele, U., & Schneider, S. (2001). Predictors of reading literacy. European Journal of Psychology of Education, 16(3), 262–383.
Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students’ learning with hypermedia? Journal of Educational Psychology, 96(3), 523–535.
Azevedo, R., Cromley, J. G., & Seibert, D. (2004). Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia? Contemporary Educational Psychology, 29, 344–370.
Baker, L., & Cerro, L. C. (2000). Assessing metacognition in children and adults. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 99–146). Lincoln, NE: Buros Institute.
Bråten, I., & Samuelstuen, M. S. (2004). Does the influence of reading purpose on reports of strategic text processing depend on students’ topic knowledge? Journal of Educational Psychology, 96(2), 324–336.
Brown, A. L., & Day, J. D. (1983). Macrorules for summarizing texts: The development of expertise. Journal of Verbal Learning and Verbal Behavior, 22(1), 1–14.
Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental validation of transactional strategies instruction with low-achieving second-grade readers. Journal of Educational Psychology, 88(1), 18–37.
Crain-Thoreson, C., Lippman, M. Z., & McClendon-Magnuson, D. (1997). Windows on comprehension: Reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology, 89(4), 579–591.
Cromley, J. G. (2005). Reading comprehension component processes in early adolescence.Dissertation, University of Maryland, College Park, MD.
Cromley, J. G., & Azevedo, R. (2004a). Testing the fit of three models of reading comprehension with 9th grade students. Poster presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Cromley, J. G., & Azevedo, R. (2004b). Using think-aloud data to illuminate a model of high school reading comprehension. Poster presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Cromley, J. G., & Azevedo, R. (2005). Testing the fit of four variations of the Inferential Mediation model of reading comprehension. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Quebec, CA.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised edition). Cambridge, MA: Bradford.
Ericsson, K. A., & Simon, H. A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, & Activity, 5(3), 178–186.
Garcia, G. E. (1991). Factors influencing the English reading test performance of Spanish-speaking Hispanic children. Reading Research Quarterly, 25(4), 371–392.
Garner, R. (1987). Metacognition and reading comprehension. Norwood, NJ: Ablex.
Hacker, D. J. (1998). Self-regulated comprehension during normal reading. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 165–191). Mahwah, NJ: Erlbaum.
Hadwin, A., Winne, P., Stockley, D., Nesbit, J., & Woszczyna, C. (2001). Context moderates students’ self-reports about how they study. Journal of Educational Psychology, 93(3), 477–487.
Kozminsky, E., & Kozminsky, L. (2001). How do general knowledge and reading strategies ability relate to reading comprehension of high school students at different educational levels? Journal of Research in Reading, 24(2), 187–204.
MacGinitie, W. H., MacGinitie, R. K., Maria, K., & Dreyer, L. G. (2001). Gates–MacGinitie reading tests, Level 7/9, Form S (4th ed.). Itasca, IL: Riverside.
Magliano, J. P., & Graesser, A. C. (1991). A three-pronged method for studying inference generation in literary text. Poetics, 20, 193–232.
McNamara, D. S. (2004). SERT: Self-Explanation Reading Training. Discourse Processes, 38(1), 1–30.
Mokhtari, K., & Reichard, C. A. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94(2), 249–259.
National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: NICHD.
Nokes, J. D., & Dole, J. A. (2004). Helping adolescent readers through explicit strategy instruction. In T. L. Jetton & J. A. Dole (Eds.), Adolescent literacy research and practice (pp. 162–182). NY: Guilford.
Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 2, 117–175.
Paris, S. G., Cross, D. R., & Lipson, M. Y. (1984). Informed strategies for learning: A program to improve children’s reading awareness and comprehension. Journal of Educational Psychology, 76(6), 1239–1252.
Paris, S. G., Wasik, B. A., & Turner, J. C. (1991). The development of strategic readers. In R. Barr, M. L. Kamil, P. B. Mosenthal, & P. D. Pearson (Eds.), Handbook of reading research, volume II (pp. 609–640). New York: Longman.
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.
Pressley, M., & Hilden, K. (2004). Verbal protocols of reading. In N. K. Duke & M. H. Mallette (Eds.), Literacy research methodologies (pp. 308–321). NY: Guilford.
Rinehart, S. D., Stahl, S. A., & Erickson, L. G. (1986). Some effects of summarization training on reading and studying. Reading Research Quarterly, 21(4), 422–438.
Rogers, T. (1991). Students as literary critics: The interpretive experiences, beliefs, and processes of ninth-grade students. Journal of Reading Behavior, 23(4), 391–423.
Roller, C. M. (1986). Overcoming underwriting and other textbook sins. Social Education, 50(1), 56–57.
Smith, R. S. W., Roehrig, A. D., & Guan, Q. (2006, April). Validation of a tool to assess the metacognitive awareness of reading strategies within adolescent and adult readers. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51–79.
Sperling, R. A., Howard, B. C., Staley, R., & DuBois, N. (2004). Metacognition and self-regulated learning constructs. Educational Research and Evaluation, 10(2), 117–139.
Stamp Act. (2004). Britannica Student Encyclopedia. Downloaded from the World Wide Web from http://www.search.eb.com/.
Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In B. Moschner & C. Artelt (Eds.), Lernstrategien und Metakognition: Implikationen für Forschung und Praxis (pp. 75–97). Berlin: Waxmann.
Veenman, M. V. J., Elshout, J. J., & Groen, M. G. M. (1993). Thinking aloud: Does it affect regulatory processes in learning. Tijdschrift voor Onderwijsresearch, 18, 322–330.
Viola, H. J., Wheatley, H., & Hart, D. (1998). Why we remember. Menlo Park, CA: Addison-Wesley.
Wyatt, D., Pressley, M., El-Dinary, P. B., Stein, S., Evans, P., & Brown, R. (1993). Comprehension strategies, worth and credibility monitoring, and evaluations: Cold and hot cognition when experts read professional articles that are important to them. Learning and Individual Differences, 5(1), 49–72.
This research was supported by funding from an AERA/Spencer Pre-Dissertation Fellowship and a Spencer Dissertation Fellowship to the first author and by funding from the National Science Foundation (REC#0133346) awarded to the second author. Portions of this research were previously presented at the Annual Meeting of the American Educational Research Association, April 15, 2005, Montreal, Quebec, Canada. We thank Dr. Sheila Barron, Jeffrey Greene, Dr. Andrew Ho, Dr. Lorena Llosa, and Dr. Jean-Pierre Verhaege for comments that improved the quality of this paper.
Major topics, sub-topics, and supporting details used for scoring the free recalls
|Major topics (4 points each)||Sub-topics (2 points each)||Supporting details (1 point each)|
|To fund army||French and Indian War|
|Sugar Act||halved/dropped [tax]||1760s|
|duties/tax [on molasses]|
|enforce||writs of assistance|
|guilty until proven innocent|
|No taxation without representation||no representatives||Parliament|
|1,000s of miles away|
|Stamp Act||buy tax stamps [not just special stamp]||tax stamp|
|on paper||legal documents|
|affected more people|
|disagreements in Parliament||Barre|
|Stamp Act Congress||nine colonies|
|Sons of Violence|
|burn [effigy or paper]|
|[CT] tax collector|
|started to bury|
|Sons and Daughters of Liberty|
Sample items from the self-reported strategy use and multiple-choice strategy use measures
Self-reported Strategy Use (MARSI; Mokhtari & Reichard, 2002, p. 259)
‘I never or almost never do this.’
‘I do this only occasionally.’
‘I sometimes do this.’ (about 50% of the time)
‘I usually do this.’
‘I always or almost always do this.’
“6. I summarize what I read to reflect on important information in the text 1 2 3 4 5 ”
Multiple-choice strategy use
A Stamp Act Congress, representing nine colonies, met in New York City on Oct. 7, 1765. The congress declared that only the colonial assemblies should tax the colonists. The congress also petitioned the king and Parliament for repeal of the objectionable measures. When the stamped papers began to arrive, mobs seized them or forced the ships’ captains to take them back to England.
Which of the following is the best summary of the passage?
The Stamp Act Congress met in 1765
Mobs seized stamped papers from English ships
The Stamp Act Congress met and asked Parliament to repeal the Act
Nine colonies met and petitioned the King
Many wealthy merchants favored stopping all business that required the use of stamped papers. This, they said, would be perfectly legal. They also argued that it would so seriously interfere with the business of British merchants that Parliament would be forced to repeal the law.
Which sentence would it make the most sense to underline or highlight?
Many wealthy merchants favored stopping all business...
This, they said, would be...
They also argued that it would so seriously interfere...
All of the above.
Strategy codes for the think-aloud measure (Adapted from Azevedo & Cromley, 2004)
|HYP+||Any hypothesis or prediction about events to follow in the text (could be an accurate, sensible, or even insensible hypothesis/prediction).||“They might start, like, charging people for the soldiers.”|
|INF+||Makes an accurate within-text inference. The word “because” always signals an inference, but not all inferences include a causal word.||“Because they had a lot of goods to sell.”|
|PKA+||Activates prior knowledge that is both accurate and relevant.||“But didn’t Britain like force them to take it? Yeah.”|
|SQ+||Participant poses him/herself a question that might potentially be answered by the forthcoming text; not a rhetorical question; need not include a question word; not a misunderstanding of text that came before; not a statement of lack of understanding; not PKA-.||“I’m thinking how big is the army then if they needed 10,000 soldiers just for North America.”|
|SUM+||Accurately summarizes (note that part of a summary can be accurate and part can be inaccurate—2 codes).||“to make people pay for their own defenses.”|
|INF-||Makes an inaccurate within-text inference.||“So they were doing illegal acts in order to raise money.”|
|PKA-||Activates inaccurate and/or irrelevant prior knowledge or states that he/she lacks background knowledge.||“That rum was like prohibited, like in those times.”|
|SUM-||Inaccurately summarizes (note that part of a summary can be inaccurate and part can be accurate—2 codes). Could misunderstand or over-generalize in the summary; not PKA-.||“They were selling their things.”|
About this article
Cite this article
Cromley, J.G., Azevedo, R. Self-report of reading comprehension strategies: What are we measuring?. Metacognition Learning 1, 229–247 (2006). https://doi.org/10.1007/s11409-006-9002-5