Instructional Science

, Volume 47, Issue 6, pp 627–658 | Cite as

Studying the expertise reversal of the multimedia signaling effect at a process level: evidence from eye tracking

  • Juliane RichterEmail author
  • Katharina Scheiter
Original Research


The purpose of this study was to shed light on the cognitive processes underlying the expertise reversal effect related to multimedia signaling. Multimedia signals highlight correspondences between text and pictures, which is supposed to support text-picture integration and thus learning from multimedia. Previous research suggests that learners’ prior knowledge moderates the multimedia signaling effect in that they only aid learners with low prior knowledge (LPK). We conducted an eye tracking study with students in secondary education who learned with a digital textbook in one of the two versions: (a) a basic version with mostly text signals only (e.g., bold face), or (b) an extended version with additional multimedia signals that aimed at supporting text-picture integration (e.g., color coding of corresponding text and picture elements). In addition to learning outcomes, we assessed students’ cognitive load and gaze behavior as process measures. Results revealed that only LPK learners were supported in learning whereas HPK learners were not affected by multimedia signals (partial expertise reversal). A moderated mediation analysis revealed that multimedia signals affected gaze behavior of LPK students in that they looked earlier at pictures. For high prior knowledge students multimedia signals lead to a higher subjective germane cognitive load. Thus, multimedia signals affected processing of materials. However, the process measures did not explain the expertise reversal of the signaling effect regarding learning outcome.


Expertise reversal effect Multimedia learning Signaling Prior knowledge Eye tracking 



This work was supported by the German Research Foundation (DFG) under contract SCH683/6.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Arslan-Ari, I. (2017). Learning from instructional animations: How does prior knowledge mediate the effect of visual cues? Journal of Computer Assisted Learning,34, 140–149. Scholar
  2. Attar, N., Schneps, M. H., & Pomplun, M. (2016). Working memory load predicts visual search efficiency: Evidence from a novel pupillary response paradigm. Memory & Cognition,44, 1038–1049. Scholar
  3. Aydeniz, M., & Kotowski, E. L. (2012). What do middle and high school students know about the particulate nature of matter after instruction? Implications for practice. School Science and Mathematics,112, 59–65. Scholar
  4. Baumert, J., Lehmann, R., Lehrke, M., Clausen, M., Hosenfeld, I., Neubrand, J., … Günther, W. (1998). Testaufgaben Naturwissenschaften TIMSS 7./8. Klasse (Population 2). Max-Planck-Institut für Bildungsforschung.Google Scholar
  5. Butcher, K. R. (2006). Learning from text with diagrams: Promoting mental model development and inference generation. Journal of Educational Psychology,98, 182–197. Scholar
  6. Butcher, K. R. (2014). The multimedia principle. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 174–205). New York: Cambridge University Press.CrossRefGoogle Scholar
  7. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction,8, 293–332. Scholar
  8. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science,5, 121–152. Scholar
  9. Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior,25, 315–324. Scholar
  10. De Groot, S. G., & Gebhard, J. W. (1952). Pupil size as determined by adapting luminance. Journal of the Optical Society of America,42, 492–495. Scholar
  11. De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science,38, 105–134. Scholar
  12. Eitel, A., Scheiter, K., & Schüler, A. (2013). How inspecting a picture affects processing of text in multimedia learning. Applied Cognitive Psychology,27, 451–461. Scholar
  13. Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011). Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review,23, 523–552. Scholar
  14. Grüß-Niehaus, T. (2010). Comprehension of the dissolution concept in chemistry education. Hannover: Gottfried Wilhelm Leibniz Universität. (in German).Google Scholar
  15. Haider, H., & Frensch, P. A. (1999). Information reduction during skill acquisition: The influence of task instruction. Journal of Experimental Psychology: Applied,5, 129–151. Scholar
  16. Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of science textbook passages among low-and high-ability children. Contemporary Educational Psychology,24, 95–123. Scholar
  17. Hollstein, A. (2001). Computer-supported learning on the basis of constructivist learning theories through the example of the introduction of the particle model. Essen: University of Duisburg-Essen. (in German).Google Scholar
  18. Homer, B. D., & Plass, J. L. (2009). Expertise reversal for iconic representations in science visualizations. Instructional Science,38, 259–276. Scholar
  19. Jamet, E. (2014). An eye-tracking study of cueing effects in multimedia learning. Computers in Human Behavior,32, 47–53. Scholar
  20. Judd, C. M., Yzerbyt, V. Y., & Muller, D. (2014). Mediation and moderation. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in personality and social psychology (2nd ed., pp. 653–676). New York: Cambridge University Press. Scholar
  21. Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review,87, 329–354. Scholar
  22. Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review,19, 509–539. Scholar
  23. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist,38, 23–31. Scholar
  24. Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors: The Journal of the Human Factors and Ergonomics Society,40, 1–17. Scholar
  25. Kalyuga, S., & Plass, J. L. (2018). Cognitive load as a local characteristic of cognitive processes: Implications for measurement approaches. In R. Z. Zheng (Ed.), Cognitive load measurement and application: A theoretical framework for meaningful research and practice (pp. 59–74). New York: Routledge.Google Scholar
  26. Kalyuga, S., & Renkl, A. (2010). Expertise reversal effect and its instructional implications: Introduction to the special issue. Instructional Science,38, 209–215. Scholar
  27. Klepsch, M., Schmitz, F., & Seufert, T. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in Psychology. Scholar
  28. Lemarié, J., Lorch, R. F. J., Eyrolle, H., & Virbel, J. (2008). SARA: A text-based and reader-based theory of signaling. Educational Psychologist,43, 27–48. Scholar
  29. Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods,45, 1058–1072. Scholar
  30. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods,7, 83–104. Scholar
  31. Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers & Education,60, 95–109. Scholar
  32. Mautone, P. D., & Mayer, R. E. (2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology,93, 377–389. Scholar
  33. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.CrossRefGoogle Scholar
  34. Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd ed.). New York: Cambridge University Press.CrossRefGoogle Scholar
  35. McNamara, D., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction,14, 1–43. Scholar
  36. Meyer, B. J. F. (1975). The organization of prose and its effects on memory. Amsterdam: North-Holland Publishing.Google Scholar
  37. Mulford, D. R., & Robinson, W. R. (2002). An inventory for alternate conceptions among first-semester general chemistry students. Journal of Chemical Education,79, 739–744. Scholar
  38. Muller, D., Judd, C. M., & Yzerbyt, V. Y. (2005). When moderation is mediated and mediation is moderated. Journal of Personality and Social Psychology,89, 852–863. Scholar
  39. Nakhleh, (1992). Why some students don’t learn chemistry: Chemical misconceptions. Journal of Chemical Education,69, 191–196. Scholar
  40. Nückles, M., Hübner, S., Dümer, S., & Renkl, A. (2010). Expertise reversal effects in writing-to-learn. Instructional Science,38, 237–258. Scholar
  41. Nunnally, J. C., Knott, P. D., Duchnowski, A., & Parker, R. (1967). Pupillary response as a general measure of activation. Perception & Psychophysics,2, 149–155. Scholar
  42. Oksa, A., Kalyuga, S., & Chandler, P. (2010). Expertise reversal effect in using explanatory notes for readers of Shakespearean text. Instructional Science,38, 217–236. Scholar
  43. Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior,26, 110–117. Scholar
  44. Ozcelik, E., Karakus, T., Kursun, E., & Cagiltay, K. (2009). An eye-tracking study of how color coding affects multimedia learning. Computers & Education,53, 445–453. Scholar
  45. Paas, F., Tuovinen, J. E., Tabbers, H. K., & van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist,38, 63–71. Scholar
  46. Petermann, K., Friedrich, J., & Oetken, M. (2009). Test zur Diagnose von Schülervorstellungen zum Teilchenkonzept. Praxis der Naturwissenschaften—Chemie in der Schule, 58, 41–43.Google Scholar
  47. Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research,42, 185–227. Scholar
  48. Renkl, A., & Scheiter, K. (2017). Studying visual displays: How to instructionally support learning. Educational Psychology Review,29, 599–621. Scholar
  49. Richter, J., Scheiter, K., & Eitel, A. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17, 19–36. Scholar
  50. Richter, J., Scheiter, K., & Eitel, A. (2018). Signaling text–picture relations in multimedia learning: The influence of prior knowledge. Journal of Educational Psychology, 110, 544–560. Scholar
  51. Rop, G., Schüler, A., Verkoeijen, P. P. J. L., Scheiter, K., & van Gog, T. (2018). Effects of task experience and layout on learning from text and pictures with or without unnecessary picture descriptions. Journal of Computer Assisted Learning,34, 458–470. Scholar
  52. Salomon, G. (1984). Television is “easy” and print is “tough”: The differential investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology,76, 647–658. Scholar
  53. Schanze, S. (2002). Knowledge acquisition by means of the online learning environment ChemNet. Kiel: Christian-Albrechts-Universität zu Kiel, Germany. (in German).Google Scholar
  54. Scharinger, C., Kammerer, Y., & Gerjets, P. (2015). Pupil dilation and EEG alpha frequency band power reveal load on executive functions for link-selection processes during text reading. PLoS ONE,10, 1–24. Scholar
  55. Scheiter, K., & Eitel, A. (2015). Signals foster multimedia learning by supporting integration of highlighted text and diagram elements. Learning and Instruction,36, 11–26. Scholar
  56. Scheiter, K., & Eitel, A. (2016). The use of eye tracking as a research and instructional tool in multimedia learning. In C. Was, F. Sansosti, & B. Morris (Eds.), Eye-tracking technology applications in educational research (pp. 143–164). Hershey, PA: IGI Global.Google Scholar
  57. Schmeck, A., Opfermann, M., van Gog, T., Paas, F., & Leutner, D. (2015). Measuring cognitive load with subjective rating scales during problem solving: Differences between immediate and delayed ratings. Instructional Science,43, 93–114. Scholar
  58. Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010). A closer look at split visual attention in system-and self-paced instruction in multimedia learning. Learning and Instruction,20, 100–110. Scholar
  59. Schneider, S., Beege, M., Nebel, S., & Rey, G. D. (2018). A meta-analysis of how signaling affects learning with media. Educational Research Review,23, 1–24. Scholar
  60. Schneider, W., Schlagmüller, M., & Ennemoser, M. (2007). Reading speed and comprehension test for grades 6–12. Göttingen: Hogrefe. (in German).Google Scholar
  61. Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review,84, 1–66. Scholar
  62. Schnotz, W. (2010). Reanalyzing the expertise reversal effect. Instructional Science,38, 315–323. Scholar
  63. Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review,19, 469–508. Scholar
  64. Schwonke, R., Berthold, K., & Renkl, A. (2009a). How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology,23, 1227–1243. Scholar
  65. Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., & Salden, R. (2009b). The worked-example effect: Not an artefact of lousy control conditions. Computers in Human Behavior,25, 258–266. Scholar
  66. Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology,20, 321–331. Scholar
  67. Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. Psychological Review,84, 127–190. Scholar
  68. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Expertise reversal effect. In Cognitive Load (Ed.), Theory: Explorations in the learning sciences, instructional systems and performance technologies (pp. 155–170). New York: Springer.Google Scholar
  69. Sweller, J., van Merriënboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review,10, 251–296. Scholar
  70. Van Gerven, P. W. M., Paas, F., Van Merriënboer, J., & Schmidt, H. G. (2004). Memory load and the cognitive pupillary response in aging. Psychophysiology,41, 167–174. Scholar
  71. Van Gog, T. (2014). The signaling (or cueing) principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 263–278). New York: Cambridge University Press.Google Scholar
  72. Wilde, M., Bätz, K., Kovaleva, A., & Urhahne, D. (2009). Testing a short scale of intrinsic motivation. Zeitschrift für Didaktik der Naturwissenschaften,15, 31–45. (in German).Google Scholar
  73. Yezierski, E. J., & Birk, J. P. (2006). Misconceptions about the particulate nature of matter. Using animations to close the gender gap. Journal of Chemical Education,83, 954–960. Scholar
  74. ZPG-Chemie (2011, Febuary). Das Teilchenmodell in Klasse 8 [The Particle Model of Matter in grade 8]. Retrieved February 28, 2016, from Landesakademie für Fortbildung und Personalentwicklung an Schulen website,

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Leibniz-Institut für WissensmedienTübingenGermany
  2. 2.University of TübingenTübingenGermany

Personalised recommendations