Assessing Representational Competence with Eye Tracking Technology

  • Inga UbbenEmail author
  • Sandra Nitz
  • Kristy L. Daniel
  • Annette Upmeier zu Belzen
Part of the Models and Modeling in Science Education book series (MMSE, volume 11)


Although it may appear trivial, the first step in developing representational competence is literally looking at a representation. This chapter focuses on eye tracking technology as a tool for assessing visual attention while using representations, particularly with regard to understanding the underlying cognitive processes of representational competence. This technology is not new, but its use is expanding in science education. We give an overview of how eye tracking technology works, what it can measure, and how this type of data can be used as evidence for representation use. In combination with verbal and written data, eye tracking technology might be able to more finely distinguish between novices and experts in the visual use of representations and capture levels of representational competence. We synthesize what has been learned from past uses of this technology in science education and provide insights for potential future uses as an assessment of representational competence to help further this field.


Representational Competence (RC) Areas Of Interest (AOIs) Scanpath Jarodzka Representational Forms 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Baum, D. A., DeWitt Smith, S., & Donovan, S. S. S. (2005). The tree-thinking challenge. Science, 310(5750), 979–980.CrossRefGoogle Scholar
  2. Boucheix, J.-M., & Lowe, R. K. (2010). An eye-tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, 20(2), 123–135.CrossRefGoogle Scholar
  3. Brinkman, J. A. (1993). Verbal protocol accuracy in fault diagnosis. Ergonomics, 36(11), 1381–1397.CrossRefGoogle Scholar
  4. Canham, M., & Hegarty, M. (2010). Effects of knowledge and display design on comprehension of complex graphics. Learning and Instruction, 20(2), 155–166.CrossRefGoogle Scholar
  5. Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152.CrossRefGoogle Scholar
  6. Cook, M., Wiebe, E. N., & Carter, G. (2008). The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations. Science Education, 92(5), 848–867.CrossRefGoogle Scholar
  7. Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455–470.CrossRefGoogle Scholar
  8. Duchowski, A. T. (2007). Eye tracking methodology: Theory and practice (2nd ed.). London: Springer. With permission of Springer.Google Scholar
  9. Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211–245.CrossRefGoogle Scholar
  10. Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptations to task constraints. Annual Review of Psychology, 47, 273–305.CrossRefGoogle Scholar
  11. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press.Google Scholar
  12. Feusner, M., & Lukoff, B. (2008). Testing for statistically significant differences between groups of scan patterns. In S. N. Spencer (Ed.), Proceedings of the 2008 symposium on eye tracking research & applications (pp. 43–46). New York: ACM.CrossRefGoogle Scholar
  13. Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011). Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review, 23(4), 523–552.CrossRefGoogle Scholar
  14. Gilbert, S. W. (1991). Model building and a definition of science. Journal of Research in Science Teaching, 28(1), 73–79.CrossRefGoogle Scholar
  15. van Gog, T., Paas, F., Merriënboer, v., Jeroen, J. G., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology. Applied, 11(4), 237–244.CrossRefGoogle Scholar
  16. van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25(3), 785–791.CrossRefGoogle Scholar
  17. Grünkorn, J., Upmeier zu Belzen, A., & Krüger, D. (2013). Assessing Students’ understandings of biological models and their use in science to evaluate a theoretical framework. International Journal of Science Education, 36(10), 1651–1684.CrossRefGoogle Scholar
  18. Haider, H., & Frensch, P. A. (1999). Eye movement during skill acquisition: More evidence for the information reduction hypothesis. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 172–190.Google Scholar
  19. Halverson, K. L. (2011). Improving tree-thinking one learnable skill at a time. Evolution: Education and Outreach, 4(1), 95–106.Google Scholar
  20. Halverson, K. L., & Friedrichsen, P. (2013). Learning tree thinking: Developing a new framework of representational competence. In D. F. Treagust & C.-Y. Tsui (Eds.), Models and modeling in science education: Vol. 7, Multiple Representations in Biological Education (pp. 185–201). Dordrecht: Springer.Google Scholar
  21. Halverson, K. L., Pires, C. J., & Abell, S. K. (2011). Exploring the complexity of tree thinking expertise in an undergraduate systematics course. Science Education, 95(5), 794–823.CrossRefGoogle Scholar
  22. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford, New York: Oxford University Press.Google Scholar
  23. Jarodzka, H., Scheiter, K., Gerjets, P., & van Gog, T. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction, 20(2), 146–154.CrossRefGoogle Scholar
  24. Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter, K., Gerjets, P., & Eika, B. (2012). Conveying clinical reasoning based on visual observation via eye-movement modeling examples. Instructional Science, 40(5), 813–827.CrossRefGoogle Scholar
  25. Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students’ attention via a Model's eye movements fosters learning. Learning and Instruction, 25, 62–70.CrossRefGoogle Scholar
  26. Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.CrossRefGoogle Scholar
  27. Koning, d., Björn, B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2010). Attention guidance in learning from a complex animation: Seeing is understanding? Learning and Instruction, 20(2), 111–122.CrossRefGoogle Scholar
  28. Kozma, R., & Russell, J. (1997). Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena. Journal of Research in Science Teaching, 34(9), 949–968.CrossRefGoogle Scholar
  29. Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In J. K. Gilbert (Ed.), Visualization in science education (pp. 121–145). Dordrecht: Springer.CrossRefGoogle Scholar
  30. Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemistry laboratory and their implications for chemistry learning. The Journal of the Learning Sciences, 9(2), 105–143.CrossRefGoogle Scholar
  31. Kundel, H. L., Nodine, C. F., Conant, E. F., & Weinstein, S. P. (2007). Holistic component of image perception in mammogram interpretation: Gaze-tracking study. Radiology, 242, 396–402.CrossRefGoogle Scholar
  32. Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y., et al. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90–115.CrossRefGoogle Scholar
  33. Mahr, B. (2008). Ein Modell des Modellseins: Ein Beitrag zur Aufklärung des Modellbegriffs. In E. Knobloch & U. Dirks (Eds.), Modelle (pp. 187–218). Frankfurt am Main: Peter Lang.Google Scholar
  34. Mahr, B. (2009). Die Informatik und die Logik der Modelle. Informatik Spektrum, 32(3), 228–249.CrossRefGoogle Scholar
  35. van Marlen, T., van Wermeskerken, M., Jarodzka, H., & van Gog, T. (2016). Showing a model's eye movements in examples does not improve learning of problem-solving tasks. Computers in Human Behavior, 65, 448–459.CrossRefGoogle Scholar
  36. Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, 172–187.CrossRefGoogle Scholar
  37. McMains, S. A., & Kastner, S. (2009). Visual Attention. In M. D. Binder, N. Hirokawa, & U. Windhorst (Eds.), Encyclopedia of neuroscience (pp. 4296–4302). Berlin, Heidelberg: Springer.CrossRefGoogle Scholar
  38. Nersessian, N. J. (2002). The cognitive basis of model-based reasoning in science. In The cognitive basis of science (pp. 133–153).CrossRefGoogle Scholar
  39. Nitz, S., Ainsworth, S. E., Nerdel, C., & Prechtl, H. (2014). Do student perceptions of teaching predict the development of representational competence and biological knowledge? Learning and Instruction, 31, 13–22.CrossRefGoogle Scholar
  40. Novick, L. R., & Catley, K. M. (2007). Understanding phylogenies in biology: The influence of a gestalt perceptual principle. Journal of Experimental Psychology: Applied, 13(4), 197–223.Google Scholar
  41. Novick, L. R., & Catley, K. M. (2014). When relationships depicted diagrammatically conflict with prior knowledge: An investigation of students’ interpretations of evolutionary trees. Science Education, 98(2), 269–304.CrossRefGoogle Scholar
  42. Novick, L. R., Stull, A. T., & Catley, K. M. (2012). Reading phylogenetic trees: The effects of tree orientation and text processing on comprehension. Bioscience, 62(8), 757–764.CrossRefGoogle Scholar
  43. O'Hara, R. J. (1988). Homage to Clio, or, toward an historical philosophy for evolutionary biology. Systematic Zoology, 37(2), 142–155.CrossRefGoogle Scholar
  44. O'Hara, R. J. (1997). Population thinking and tree thinking in systematics. Zoologica Scripta, 26(4), 323–329.CrossRefGoogle Scholar
  45. Omland, K. E., Cook, L. G., & Crisp, M. D. (2008). Tree thinking for all biology: The problem with reading phylogenies as ladders of progress. BioEssays, 30(9), 854–867.CrossRefGoogle Scholar
  46. Passmore, C., Gouvea, J. S., & Giere, R. (2014). Models in science and in learning science: Focusing scientific practice on sense-making. In M. R. Matthews (Ed.), International handbook of research in history, philosophy and science teaching (1st ed., pp. 1171–1202). Dordredht: Springer Netherlands.Google Scholar
  47. Rayner, K. (1998). Eye movements in reading and information processing. 20 years of research. Psychological Bulletin, 124(3), 372–422.CrossRefGoogle Scholar
  48. Rayner, K., Pollatsek, A., Ashby, J., & Clifton, C., Jr. (2012). Psychology of reading. New York: Psychology Press.Google Scholar
  49. van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical guide to modeling cognitive processes. London: Academic Press.Google Scholar
  50. Stieff, M. (2007). Mental rotation and diagrammatic reasoning in science. Learning and Instruction, 17(2), 219–234.CrossRefGoogle Scholar
  51. Stieff, M., Hegarty, M., & Deslongchamps, G. (2011). Identifying representational competence with multi-representational displays. Cognition and Instruction, 29(1), 123–145.CrossRefGoogle Scholar
  52. Tippett, C. D., & Yore, L. (2011). Exploring middle school students’ representational competence in science: Development and verification of a framework for learning with visual representations.Google Scholar
  53. Upmeier zu Belzen, A., & Krüger, D. (2010). Modellkompetenz im Biologieunterricht. ZfDN, 16, 41–57.Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Inga Ubben
    • 1
    Email author
  • Sandra Nitz
    • 2
  • Kristy L. Daniel
    • 3
  • Annette Upmeier zu Belzen
    • 1
  1. 1.Humboldt-Universität zu BerlinBerlinGermany
  2. 2.University of Koblenz and LandauLandauGermany
  3. 3.Texas State UniversitySan MarcosUSA

Personalised recommendations