Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: an eye-movement study
- 749 Downloads
Printed or digital textbooks contain texts accompanied by various kinds of visualisation. Successful comprehension of these materials requires integrating verbal and graphical information. This study investigates the time course of processing an illustrated text through eye-tracking methodology in the school context. The aims were to identify patterns of first- and second-pass reading and to examine whether the integrative processing of text and picture during the less automatic and more purposeful second-pass reading predicts learning, after controlling for reading comprehension, prior knowledge, and self-concept. Forty-three 7th graders read an illustrated science text while their eye movements were recorded. A cluster analysis revealed two processing patterns during the first-pass reading, which differed for the time spent on the main concepts in the text and picture. During re-reading, two patterns of stronger and weaker integrative processing emerged. Integration of verbal and graphical information was revealed by the frequency of second-pass transitions from text to picture and from picture to text, and the duration of picture re-inspecting while re-reading text information (look-from text to picture) and re-reading text information while re-inspecting the visualised information (look-from picture to text). A series of hierarchical regression analyses indicated that only the patterns of integrative processing during the second-pass reading uniquely predict verbal and graphical recalls, and the transfer of knowledge. The study provides evidence that the delayed processing which integrates text and graphics contributes to text retention and the application of newly learned knowledge, over and above individual characteristics. The educational significance is outlined.
KeywordsText processing Reading comprehension Integrative processing Multimedia learning Eye movements
- Bartholomé, T., & Bromme, R. (2009). Coherence formation when learning from text and pictures: What kind of support for whom? Journal of Educational Psychology, 101(2), 282–293. doi: 10.1037/a0014312.
- Butcher, K. R. (2006). Learning from text with diagrams: Promoting mental model development and inference generation. Journal of Educational Psychology, 98, 182–197. doi: 10.1037/0022-0618.104.22.168
- Cornoldi, C., & Colpo, G. (1995). Nuove prove MT per la scuola media [New MT tests of reading comprehension for the middle school] Florence. Italy: Organizzazioni Speciali.Google Scholar
- Eitel, A., Scheitel, K. (2014). Picture or text first? Explaining sequence effects when learning with pictures and text. Educational Psychology Review. doi:0.1007/s10648-014-9264-4Google Scholar
- Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742. doi: 10.1006/jmla.1993.1036
- Mason, L., Boscolo, P., Tornatora, M. C., & Ronconi, L. (2013a). Besides knowledge: A cross-sectional study on the relations between epistemic beliefs, achievement goals, self-beliefs, and achievement in science. Instructional Science, 41(1), 49–79. doi: 10.1007/s11251-012-9210-0.CrossRefGoogle Scholar
- Mikkilä-Erdmann, M., Penttinen, M., Anto, E., & Olkinuora, E. (2008). Constructing mental models during learning from science text. Eye tracking methodology meets conceptual change. In D. Ifenthaler, P. Pirnay-Dummer, & J. Michael Spector (Eds.), Understanding models for learning and instruction: Essays in honor of Norbert M. Seel. (pp. 63–79). New York: Springer.Google Scholar
- Schnotz, W., Ludewig, U., Ulrich, M., Horz, H., McElvany, N., & Baumert, J. (2014). Strategy shifts during learning from texts and picture. Journal of Educational Psychology. Advance online publication. doi/ 10.1037/a0037054
- Schroeder, S., Richter, T., McElvany, N., Hachfeld, A., Baumert, J., Schnotz, W., et al. (2011). Teachers’ beliefs, instructional behaviors, and students’ engagement in learning from texts with instructional pictures. Learning and Instruction, 21(3), 403–415. doi: 10.1016/j.learninstruc.2010.06.001.CrossRefGoogle Scholar
- Schunk, D. A., & Pajares, F. (2005). Competence perceptions. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of competence and motivation (pp. 85–104). New York: Guilford Press.Google Scholar
- Stalbovs, K., Eitel, A., & Scheiter, K. (2013). Which cognitive processes predict successful learning with multimedia? A comparison of eye tracking parameters. Paper presented at the 15th conference of the European Association for Research on Learning and Instruction, Munich, Germany.Google Scholar
- Trautwein, U., Lüdtke, O., Köller, O., & Baumert, J. J. (2006). Self-esteem, academic self-concept, and achievement: How the learning environment moderates the dynamics of self-concept. Journal of Personality and Social Psychology, 90(2), 334–349. doi: 10.1037/0022-3522.214.171.1244.CrossRefGoogle Scholar