Abstract
This paper discusses the usefulness of tools that enable the analysis of eye movement data in dynamic interfaces for investigating theoretical issues in the area of multimedia learning. One of these tools, GazeTracker TM, a program that links eye movement data to information about the internal computer processes and automatically combines the two for further analyses, is discussed. The functionality of the tool for studying the process of multimedia learning is illustrated with an experiment on the integration of text and pictures in a web-based lesson on instructional design. In the experiment, differences in fixation patterns between several presentation formats are investigated. It is concluded that tools like GazeTracker TM make it easier to study how people integrate text and pictures in dynamic interfaces like web browsers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Baddeley, A. D. (1992). Working memory. Science, 255, 556–559.
Borg, G., Bratfisch, O., & Dornic, S. (1971). On the problem of perceived difficulty. Scandinavian Journal of Psychology, 12, 249–260.
Carroll, P. J., Young, J. R., & Guertin, M. S. (1992). Visual analysis of cartoons: A view from the far side. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading (pp. 444–461). New York: Springer-Verlag.
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332.
Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246.
Crowe, E. C., & Narayanan, N. H. (2000). Comparing interfaces based on what users watch and do. In Proceedings of the Eye Tracking Research and Applications Symposium (pp. 29–36). New York: ACM Press.
d’Ydewalle, G., & Gielen, I. (1992). Attention allocation with overlapping sound, image, and text. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading (pp. 415–427). New York: Springer-Verlag.
Duffy, S. A. (1992). Eye movements and complex comprehension processes. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading (pp. 462–471). New York: Springer-Verlag.
Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics, 24, 631–645.
Goldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., & Wichansky, A. M. (2002). Eye tracking in web search tasks: Design implications. In Proceedings of the Eye Tracking Research and Applications Symposium. New York: ACM Press.
Hegarty, M. (1992a). Mental animation: Inferring motion from static displays of mechanical systems. Journal of Experimental Psychology: Learning, Memory, & Cognition, 18, 1084– 1102.
Hegarty, M. (1992b). The mechanics of comprehension and comprehension of mechanics. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading (pp. 428–443). New York: Springer-Verlag.
Hegarty, M., & Just, M. A. (1989). Understanding machines from text and diagrams. In H. Mandl & J. R. Levin (Eds.), Knowledge acquisition from text and pictures (pp. 171–194). Amsterdam: Elsevier.
Hegarty, M., & JustJust, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.
Jeung, H., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17, 329–343.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.
Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136.
Lankford, C. (2000a). Effective eye-gaze input into WindowsTM. In Proceedings of the Eye Tracking Research and Applications Symposium. New York: ACM Press.
Lankford, C. (2000b). GazeTrackerTM: Software designed to facilitate eye movement analysis. In Proceedings of the Eye Tracking Research and Applications Symposium (pp. 57–63). New York: ACM Press.
Leahy, W., Chandler, P., & Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instructions. Applied Cognitive Psychology, 17, 401–418.
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93, 390–397.
Mayer, R. E., Dow, G. T., & Mayer, S. (2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95, 806–813.
Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90, 312–320.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: the role of modality and contiguity. Journal of Educational Psychology, 91, 358–368.
Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87, 319–334.
Narayanan, N. H., & Hegarty, M. (1998). On designing comprehensible interactive hypermedia manuals. International Journal of Human-Computer Studies, 48, 267–301.
Narayanan, N. H., & Hegarty, M. (2002). Multimedia design for communication of dynamic information. International Journal of Human-Computer Studies, 57, 279–315.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach. Journal of Educational Psychology, 84, 429–434.
Paas, F., Tuovinen, J. E., Tabbers, H. K., & Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63–71.
Paas, F., Van Merriënboer, J. J. G., & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79, 419–430.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422.
Rayner, K., Rotello, C. M., Stewart, A. J., Keir, J., & Duffy, S. A. (2001). Integrating text and pictorial information: Eye movements when looking at print advertisements. Journal of Experimental Psychology: Applied, 7, 219–226.
Salvucci, D. D., & Goldberg, J. H. (2000). Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the Eye Tracking Research and Applications Symposium (pp. 71–78). New York: ACM Press.
Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia: ACER Press.
Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.
Tabbers, H. K. (2002). The modality of text in multimedia instructions: Refining the design guidelines. Unpublished doctoral dissertation, Open University of the Netherlands, Heerlen.
Tabbers, H. K., Martens, R. L., & Van Merriënboer, J. J. G. (2001). The modality effect in multimedia instructions. J. D. Moore & K. Stenning (Eds.), Proceedings of the twenty-third annual conference of the Cognitive Science Society (pp. 1024–1029). Mahwah, NJ: Erlbaum.
Tabbers, H. K., Martens, R. L., & Van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74, 71–81.
Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3, 257–287.
Underwood, G., Jebbett, L., & Roberts, K. (2004). Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. Quarterly Journal of Experimental Psychology, 57A, 165–182.
Van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component instructional design model for technical training. Englewood Cliffs, NJ: Educational Technology Publications.
Van Orden, K. F., Limbert, W., Makeig, S., & Jung, T. P. (2001). Eye activity correlates of workload during a visuospatial memory task. Human Factors, 43, 111–121.
Acknowledgments
The experiment was part of a Ph.D. project that was co-funded by Aebly, a former multimedia company from 's Hertogenbosch, The Netherlands. We would like to thank Léon Sluijsmans from Hogeschool Zuyd in Heerlen for his helpful assistance in recruiting our participants.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Tabbers, H.K., Paas, F., Lankford, C., Martens, R.L., van Merriënboer, J.J. (2008). Studying Eye Movements in Multimedia Learning. In: Rouet, JF., Lowe, R., Schnotz, W. (eds) Understanding Multimedia Documents. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-73337-1_9
Download citation
DOI: https://doi.org/10.1007/978-0-387-73337-1_9
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-73336-4
Online ISBN: 978-0-387-73337-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)