The goal of this study was to examine how students process formative feedback that included corrective and elaborative information in online question-answering tasks. Skilled and less-skilled comprehenders in grade 8 read texts and answered comprehension questions. Prior to responding, students were asked to select the textual information relevant to answer each question. Students received formative feedback that included information about the answer correctness and an elaborative message, either automatically delivered or optionally accessed. Students’ actions, as well as verbal protocols during feedback processing, were recorded. The results showed that: (a) students paid more attention to the answer correctness than to elaborative feedback messages; (b) students paid more attention to feedback information after failure than after success; (c) students actively monitored the accuracy of their responses; and (d) differences between skilled and less-skilled comprehenders in processing feedback were very limited. Theoretical and practical implications of these results are discussed.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
PROLEC-SE is a standardized test for the evaluation of Primary and Secondary students’ reading processes in Spanish (Ramos and Cuetos 1999).
Ackerman, R., & Goldsmith, M. (2011). Metacognitive regulation of text learning: On screen versus on paper. Journal of Experimental Psychology: Applied, 17(1), 18–32. https://doi.org/10.1037/a0022086.
Ackerman, R., & Lauterman, T. (2012). Taking reading comprehension exams on screen or on paper? A metacognitive analysis of learning texts under time pressure. Computers in Human Behavior, 28(5), 1816–1828. https://doi.org/10.1016/j.chb.2012.04.023.
Anmarkrud, Ø., McCrudden, M. T., Bråten, I., & Strømsø, H. I. (2013). Task-oriented reading of multiple documents: Online comprehension processes and offline products. Instructional Science, 41(5), 873–894. https://doi.org/10.1007/s11251-013-9263-8.
Bohn-Gettler, C. M., & Kendeou, P. (2014). The interplay of reader goals, working memory, and text structure during reading. Contemporary Educational Psychology, 39(3), 206–219. https://doi.org/10.1016/j.cedpsych.2014.05.003.
Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), 209–222. https://doi.org/10.1037/0022-06188.8.131.52.
Cerdán, R., Gilabert, R., & Vidal-Abarca, E. (2011). Selecting information to answer questions: Strategic individual differences when searching texts. Learning and Individual Differences, 21(2), 201–205. https://doi.org/10.1016/j.lindif.2010.11.007.
Cerdán, R., Gilabert, R., & Vidal-Abarca, E. (2013). Self-generated explanations on the question demands are not always helpful. The Spanish Journal of Psychology, 16(e26), 1–11. https://doi.org/10.1017/sjp.2013.45.
Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018). Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review, 25, 23–38. https://doi.org/10.1016/j.edurev.2018.09.003.
Dunlosky, J., & Metcalfe, J. (2009). Metacognition: A textbook for cognitive, educational, life span and applied psychology. Thousand Oaks: Sage.
Farr, R., Pritchard, R., & Smitten, B. (1990). A description of what happens when an examinee takes a multiple-choice reading comprehension test. Journal of Educational Measurement, 27(3), 209–226. https://doi.org/10.1111/j.1745-3984.1990.tb00744.x.
Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344. https://doi.org/10.1037/a0021663.
Fox, J.-P., Klein Entink, R., & Timmers, C. (2014). The joint multivariate modeling of multiple mixed response sources: Relating student performances with feedback behavior. Multivariate Behavioral Research, 49(1), 54–66. https://doi.org/10.1080/00273171.2013.843441.
Fyfe, E. R., Rittle-Johnson, B., & DeCaro, M. S. (2012). The effects of feedback during exploratory mathematics problem solving: Prior knowledge matters. Journal of Educational Psychology, 104(4), 1094–1108. https://doi.org/10.1037/a0028389.
Graesser, A. C., & Murachver, T. (1985). Symbolic procedures of question answering. In A. C. Graesser & J. B. Black (Eds.), The psychology of questions (pp. 15–88). Hillsdale: Erlbaum.
Hattie, J., & Gan, M. (2011). Instruction based on feedback. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 249–271). New York: Routledge.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487.
Jordan, S. (2012). Student engagement with assessment and feedback: Some lessons from short-answer free-text e-assessment questions. Computers & Education, 58(2), 818–834. https://doi.org/10.1016/j.compedu.2011.10.007.
Kendeou, P., & O’Brien, E. J. (2016). Prior knowledge: Acquisition and revision. In P. Afflerbach (Ed.), Handbook of individual differences in reading: Reader, text and context (pp. 151–163). New York: Routledge.
Kendeou, P., McMaster, K. L., & Christ, T. J. (2016). Reading comprehension: Core components and processes. Policy Insights From the Behavioral and Brain Sciences, 3(1), 62–69. https://doi.org/10.1177/2372732215624707.
Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19(2), 158–170. https://doi.org/10.1016/j.learninstruc.2008.03.003.
Llorens, A. C., Gil, L., Vidal-Abarca, E., Martínez, T., Mañá, A., & Gilabert, R. (2011). Evaluación de la competencia lectora: la prueba de Competencia Lectora para Educación Secundaria (CompLEC). Psicothema, 23(4), 808–817.
Llorens, A. C., Vidal-Abarca, E., Cerdán, R., & Ávila, V. (2015). Does formative feedback on search behavior help students in answering comprehension questions from an available text? Infancia y Aprendizaje, 38(4), 808–841. https://doi.org/10.1080/02103702.2015.1076269.
Llorens, A. C., Vidal-Abarca, E., & Cerdán, R. (2016). Formative feedback to transfer self-regulation of task-oriented reading strategies. Journal of Computer Assisted Learning, 32(4), 314–331. https://doi.org/10.1111/jcal.12134.
Magliano, J. P., & Millis, K. K. (2003). Assessing reading skill with a think-aloud procedure and latent semantic analysis. Cognition and Instruction, 21(3), 251–283. https://doi.org/10.1207/S1532690XCI2103_02.
Maier, U., Wolf, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85–98. https://doi.org/10.1016/j.compedu.2015.12.002.
Máñez, I., Vidal-Abarca, E., & Martínez, T. (2016, July). Accuracy to select relevant text information to answer questions from a text. Paper presented at the 26th annual meeting of the Society for Text and Discourse, Kassel, Germany.
Martínez, T., Vidal-Abarca, E., Sellés, P., & Gilabert, R. (2008). Evaluation of comprehension strategies and processes: Test of comprehension processes (TCP). Infancia y Aprendizaje, 31(3), 319–332. https://doi.org/10.1174/021037008785702956.
Martínez, T., Vidal-Abarca, E., Gil, L., & Gilabert, R. (2009). On-line assessment of comprehension processes. The Spanish Journal of Psychology, 12(1), 308–319. https://doi.org/10.1017/S1138741600001700.
Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32, 99–113. https://doi.org/10.1023/B:TRUC.0000021811.66966.1d.
Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 745–783). Mahwah: Erlbaum.
Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merril, J. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 125–144). New York: Erlbaum.
Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multi-media learning. In H. M. Niegemann, D. Leutner, & R. Brünken (Eds.), Instructional design for multimedia learning (pp. 181–195). Münster: Waxmann.
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education, 71, 56–76. https://doi.org/10.1016/j.compedu.2013.09.011.
Ness, M. (2011). Explicit reading comprehension instruction in elementary classrooms: Teacher use of reading comprehension strategies. Journal of Research in Childhood Education, 25(1), 98–117. https://doi.org/10.1080/02568543.2010.531076.
Pekrun, R., Frenzel, A. C., Goetz, T., & Perry, R. P. (2007). The control-value theory of achievement emotions: An integrative approach to emotions in education. In P. A. Schutz & R. Pekrun (Eds.), Emotion in education (pp. 13–36). San Diego: Academic Press.
Perfetti, C. A. (1985). Reading ability. New York: Oxford University Press.
Pressley, M., Wharton-McDonald, R., Mistretta-Hampston, J., & Echevarria, M. (1998). Literacy instruction in 10 fourth-grade classrooms in upstate New York. Scientific Studies of Reading, 2(2), 159–194. https://doi.org/10.1207/s1532799xssr0202_4.
Ramos, J. L., & Cuetos, F. (1999). PROLEC-SE: Evaluación de los procesos lectores en alumnos de tercer ciclo de educación primaria y secundaria. Madrid: TEA Ediciones.
Ramos, L., & Vidal-Abarca, E. (2013). Differences between students with high and low reading literacy skills: A study with think aloud methodology. Cultura y Educación, 25(3), 295–308. https://doi.org/10.1174/113564013807749722.
Roelle, J., & Berthold, K. (2017). Effects of incorporating retrieval into learning tasks: The complexity of the tasks matters. Learning and Instruction, 49(2), 142–156. https://doi.org/10.1016/j.learninstruc.2017.01.008.
Rouet, J.-F., Britt, M. A., & Durik, A. M. (2017). RESOLV: Readers' representation of reading contexts and tasks. Educational Psychologist, 52(3), 200–215. https://doi.org/10.1080/00461520.2017.1329015.
Rupp, A. A., Ferne, T., & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective. Language Testing, 23(4), 441–474. https://doi.org/10.1191/0265532206lt337oa.
Salmerón, L., Strømsø, H. I., Kammerer, Y., Stadtler, M., & van den Broek, P. (2018). Comprehension processes in digital reading. In M. Barzillai, J. Thomson, S. Schroeder, & P. van den Broek (Eds.), Learning to read in a digital world (pp. 91–120). Amsterdam, The Netherlands: John Benjamins.
Schooler, J. W. (2011). Introspecting in the spirit of William James: Comment on Fox, Ericsson, and Best (2011). Psychological Bulletin, 137(2), 345–350. https://doi.org/10.1037/a0022390.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795.
Timmers, C. F., & Veldkamp, B. P. (2011). Attention paid to feedback provided by a computer-based assessment for learning on information literacy. Computers & Education, 56(3), 923–930. https://doi.org/10.1016/j.compedu.2010.11.007.
Timms, M., DeVelle, S., & Lay, D. (2016). Towards a model of how learners process feedback: A deeper look at learning. Australian Journal of Education, 60(2), 128–145. https://doi.org/10.1177/0004944116652912.
van den Broek, P., Kendeou, P., & White, M. J. (2009). Cognitive processes during reading: Implications for the use of multimedia to foster reading comprehension. In A. G. Bus & S. B. Neuman (Eds.), Multimedia and literacy development: Improving achievement for young learners (pp. 57–73). New York: Routledge.
Van der Kleij, F. M., Eggen, T. J. H. M., Timmers, C. F., & Veldkamp, B. P. (2012). Effects of feedback in a computer-based assessment for learning. Computers & Education, 58(1), 263–272. https://doi.org/10.1016/j.compedu.2011.07.020.
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511. https://doi.org/10.3102/0034654314564881.
Veenman, M. V. J., Elshout, J. J., & Groen, M. G. M. (1993). Thinking aloud: Does it affect regulatory processes in learning? Tijdschrift voor Onderwijsresearch, 18(6), 322–330.
Vidal-Abarca, E., Gilabert, R., & Abad, N. (2002a). A proposal for good expository text: Toward an expository text technology. Infancia y Aprendizaje, 25(4), 499–514. https://doi.org/10.1174/021037002762064064.
Vidal-Abarca, E., Reyes, H., Gilabert, R., Calpe, J., Soria, E., & Graesser, A. C. (2002b). ETAT: Expository text analysis tool. Behavior Research Methods, Instruments, & Computers, 34(1), 93–107. https://doi.org/10.3758/BF03195428.
Vidal-Abarca, E., Mañá, A., & Gil, L. (2010). Individual differences for self-regulating task-oriented reading activities. Journal of Educational Psychology, 102(4), 817–826. https://doi.org/10.1037/a0020062.
Vidal-Abarca, E., Martínez, T., Salmerón, L., Cerdán, R., Gilabert, R., Gil, L., Mañá, A., Llorens, A. C., & Ferris, R. (2011). Recording online processes in task-oriented reading with Read&Answer. Behavior Research Methods, 43(1), 179–192. https://doi.org/10.3758/s13428-010-0032-1.
Wolfe, M. B., & Goldman, S. R. (2005). Relations between adolescents' text processing and reasoning. Cognition and Instruction, 23(4), 467–502. https://doi.org/10.1207/s1532690xci2304_2.
This research was supported by the projects FPU014/04646 and EST15/00492, granted by the Spanish Ministry of Education, Culture and Sport to Ignacio Máñez, and the projects EDU2014-55662-R, granted by the Spanish Ministry of Economy and Competitiveness, and PROMETEO/2013/011, granted by Conselleria d’Educació, Cultura i Esport de la Generalitat Valenciana to the Psicotext research group at the University of Valencia.
Conflict of interest
The authors (Ignacio Máñez, Eduardo Vidal-Abarca, Panayiota Kendeou, Tomás Martínez) declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Máñez, I., Vidal-Abarca, E., Kendeou, P. et al. How do students process complex formative feedback in question-answering tasks? A think-aloud study. Metacognition Learning 14, 65–87 (2019). https://doi.org/10.1007/s11409-019-09192-w
- Formative feedback
- Metacognitive processes
- Reading comprehension