Skip to main content
SpringerLink
Log in

Temporal Methods for Eye Movement Analysis

  • Chapter
  • First Online:
Eye Movement Research

Abstract

This chapter is aimed at introducing the reader to current methods for the spatial and temporal analysis of eye movements. There are four main parts. In the first part of the chapter, we introduce the relation between attention and eye movements and then review the foundations of attention research, social attention and the effect of stimulus saliency on attention. In the second part, we first review traditional characterizations of eye movements, specifically measures of fixations and saccades, and then we review the major methods for the spatial analysis of eye movements, including heat maps and area-of-interest analyses. In the third part, we introduce a new temporal analysis of eye movements, recurrence quantification analysis. We review the basic methods and measures of this analysis, and we discuss several applications. In the last part of the chapter, we provide an overview of methods for the comparison of scanpaths and show that recurrence quantification analysis can be generalized successfully to scanpath comparisons.

With the information provided in this chapter, readers will be able to understand

  • The relationship between attention and eye movements

  • The different methods for characterizing eye movements in the spatial and temporal domain

  • The fundamentals of recurrence quantification analysis for characterizing eye movement dynamics

  • The major scanpath comparison methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

AOI:

Area of interest

CORM:

Center of recurrence mass

DET:

Determinism

LAM:

Laminarity

LDA:

Linear discriminant analysis

REC:

Recurrence

RQA:

Recurrence quantification analysis

Bibliography

  • Anderson, N. C., Anderson, F., Kingston, A., & Bischof, W. F. (2014). A comparison of scanpath comparison methods. Behavior Research Methods, https://doi.org/10.3758/s13428-014-0550-3.

    Article  Google Scholar 

  • Anderson, N. C., Bischof, W. F., Laidlaw, K. E., Risko, E. F., & Kingstone, A. (2013). Recurrence quantification analysis of eye movements. Behavior Research Methods, 45(3), 842–856.

    Article  PubMed  Google Scholar 

  • Anderson, N. C., Donk, M., & Meeter, M. (2016). The influence of a scene preview on eye movement behavior in natural scenes. Psychonomic Bulletin & Review, https://doi.org/10.3758/s13423-016-1035-4.

    Article  Google Scholar 

  • Birmingham, E., Bischof, W. F., & Kingstone, A. (2008a). Gaze selection in complex social scenes. Visual Cognition, 16, 341–355. https://doi.org/10.1080/13506280701434532.

    Article  Google Scholar 

  • Birmingham, E., Bischof, W. F., & Kingstone, A. (2008b). Social attention and real-world scenes: The roles of action, competition and social content. Quarterly Journal of Experimental Psychology, 61, 986–998. https://doi.org/10.1080/17470210701410375.

    Article  Google Scholar 

  • Birmingham, E., Bischof, W. F., & Kingstone, A. (2009). Saliency does not account for fixations to eyes within social scenes. Vision Research, 49, 2992–3000.

    Article  PubMed  Google Scholar 

  • Boccignone, G., & Ferraro, M. (2014). Ecological sampling of gaze shifts. IEEE Transactions on Cybernetics, 44(2), 266–279. https://doi.org/10.1109/TCYB.2013.2253460.

    Article  PubMed  Google Scholar 

  • Box, G. E. P., Jenkins, G. M., & Reinsel, G. C. (2008). Time series analysis (4th ed.). New York, NY: Wiley.

    Book  Google Scholar 

  • Brandt, S. A., & Stark, L. W. (1997). Spontaneous eye movements during visual imagery reflect the content of the visual scene. Journal of Cognitive Neuroscience, 9, 27–38. https://doi.org/10.1162/jocn.1997.9.1.27.

    Article  PubMed  Google Scholar 

  • Broadbent, D. E. (1971). Decision and stress. London: Academic Press.

    Google Scholar 

  • Bruce, N. D. B., & Tsotsos, J. K. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9(3), 5. https://doi.org/10.1167/9.3.5.

    Article  PubMed  Google Scholar 

  • Bunke, H. (1992). Recent advances in string matching. In H. Bunke (Ed.), Advances in structural and syntactic pattern recognition (pp. 107–116). Singapore: World Scientific.

    Google Scholar 

  • Caldara, R., & Miellet, S. (2011). iMap: A Novel Method for Statistical Fixation Mapping of Eye Movement data. Behavior Research Methods, 43(3), 864–878.

    Article  PubMed  Google Scholar 

  • Cheng, J. T., Tracy, J. L., Foulsham, T., Kingstone, A., & Henrich, J. (2013). Two ways to the top: Evidence that dominance and prestige are distinct yet viable avenues to social rank and influence. Journal of Personality and Social Psychology, 104(1), 103.

    Article  PubMed  Google Scholar 

  • Cherubini, M., Nüssli, M.-A., & Dillenbourg, P. (2010). This is it! Indicating and looking in collaborative work at distance. Journal of Eye Movement Research, 3(3), 1–20.

    Google Scholar 

  • Chica, A. B., Klein, R. M., Rafal, R. D., & Hopfinger, J. B. (2010). Endogenous saccade preparation does not produce inhibition of return: Failure to replicate Rafal, Calabresi, Brennan, & Sciolto (1989). Journal of Experimental Psychology: Human Perception and Performance, 36(5), 1193.

    PubMed  Google Scholar 

  • Chuk, T., Chan, A. B., & Hsiao, J. H. (2014). Understanding eye movements in face recognition using hidden Markov models. Journal of Vision, 14(11):8, 1–14.

    Google Scholar 

  • Coutrot, A., & Guyader, N. (2014). How saliency, faces, and sound influence gaze in dynamic social scenes. Journal of Vision, 14(8), 5.

    Article  PubMed  Google Scholar 

  • Cristino, F., Mathôt, S., Theeuwes, J., & Gilchrist, I. D. (2010). ScanMatch: A novel method for comparing fixation sequences. Behavior Research Methods, 42, 692–700. https://doi.org/10.3758/BRM.42.3.692.

    Article  PubMed  Google Scholar 

  • Dale, R., Kirkham, N. Z., & Richardson, D. C. (2011a). The dynamics of reference and shared visual attention. Frontiers in Psychology, 2(355), 1–11. https://doi.org/10.3389/fpsyg.2011.00355.

    Article  Google Scholar 

  • Dale, R., Warlaumont, A. S., & Richardson, D. C. (2011b). Nominal cross recurrence as a generalized lag sequential analysis for behavioral streams. International Journal of Bifurcation and Chaos, 21, 1153–1161. https://doi.org/10.1142/S0218127411028970.

    Article  Google Scholar 

  • Dewhurst, R., Nyström, M., Jarodzka, H., Foulsham, T., Johansson, R., & Holmqvist, K. (2012). It depends on how you look at it: Scanpath comparison in multiple dimensions with multimatch, a vector-based approach. Behavior Research Methods, 44(4), 1079–1100.

    Google Scholar 

  • Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York, NY: Chapman & Hall.

    Book  Google Scholar 

  • Findlay, J. M., & Gilchrist, I. D. (2003). Active vision: The psychology of looking and seeing. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Foulsham, T., Cheng, J. T., Tracy, J. L., Henrich, J., & Kingstone, A. (2010). Gaze allocation in a dynamic situation: Effects of social status and speaking. Cognition, 117(3), 319–331.

    Article  PubMed  Google Scholar 

  • Foulsham, T., & Kingstone, A. (2013). Fixation-dependent memory for natural scenes: An experimental test of scanpath theory. Journal of Experimental Psychology: General. https://doi.org/10.1037/a0028227.

    Article  Google Scholar 

  • Foulsham, T., & Sanderson, L. A. (2013). Look who’s talking? Sound changes gaze behaviour in a dynamic social scene. Visual Cognition, 21(7), 922–944.

    Article  Google Scholar 

  • Foulsham, T., & Underwood, G. (2008). What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. Journal of Vision, 8(2)(6), 1–17. https://doi.org/10.1167/8.2.6.

    Article  PubMed  Google Scholar 

  • Foulsham, T., Dewhurst, R., Nystrom, M., Jarodzka, H., Johansson, R., Underwood, G., & Holmqvist, K. (2012). Comparing scanpaths during scene encoding and recognition: A multi-dimensional approach. Journal of Eye Movement Research, 5(4), 1–14.

    Google Scholar 

  • Friston, K. J., Holmes, A. P., Poline, J.-B., Grasby, P. J., Williams, S. C. R., Frackowiak, R. S. J., et al. (1995). Analysis of fMRI time-series revisited. Neuroimage, 2, 45–53.

    Article  PubMed  Google Scholar 

  • Friston, K. J., Jezzard, P., & Turner, R. (1994). Analysis of functional MRI time-series. Human Brain Mapping, 1, 153–171.

    Article  Google Scholar 

  • Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.

    Google Scholar 

  • Harding, G., & Bloj, M. (2010). Real and predicted influence of image manipulations on eye movements during scene recognition. Journal of Vision, 10(2)(8), 1–17. https://doi.org/10.1167/10.2.8.

    Article  PubMed  Google Scholar 

  • Hasson, U., Yang, E., Vallines, I., Heeger, D. and Rubin, N. (2008). A hierarchy of temporal receptive windows in human cortex. Journal of Neuroscience, 28, 2539–2550.

    Article  PubMed  Google Scholar 

  • Hayes, T. R., Petrov, A. A., & Sederberg, P. B. (2011). A novel method for analyzing sequential eye movements reveals strategic influence on Raven’s Advanced Progressive Matrices. Journal of Vision, 11, 1–11.

    Google Scholar 

  • Henderson, J. M., Brockmole, J. R., Castelhano, M. S., & Mack, M. L. (2007). Visual saliency does not account for eye movements during visual search in real-world scenes. In R. van Gompel, M. Fischer, W. Murray, & R. W. Hill (Eds.), Eye movements: A window on mind and brain (pp. 537–562). Amsterdam, The Netherlands: Elsevier.

    Chapter  Google Scholar 

  • Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford, U.K.: Oxford University Press.

    Google Scholar 

  • Hunt, A. R., Chapman, C. S., & Kingstone, A. (2008). Taking a long look at action and time perception. Journal of Experimental Psychology: Human Perception and Performance, 34(1), 125.

    PubMed  Google Scholar 

  • Igarashi, T., Moscovich, T., & Hughes, J. F. (2005). Spatial keyframing for performance-driven animatoin. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, (pp. 107–115). New York: ACM.

    Google Scholar 

  • Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 1489–1506.

    Article  PubMed  Google Scholar 

  • Jarodzka, H., Holmqvist, K., & Nyström, M. (2010). A vector-based, multidimensional scanpath similarity measure. Paper presented at the Proceedings of the 2010 Symposium on Eye-Tracking Research and Applications.

    Google Scholar 

  • Josephson, S., & Holmes, M. E. (2002). Visual attention to repeated images on the World-Wide Web: Another look at scanpath theory. Behavior Research Methods, Instruments, & Computers, 34(4), 539–548.

    Article  Google Scholar 

  • Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009). Learning to predict where humans look. Paper presented at the IEEE International Conference on Computer Vision (ICCV). Kyoto, Japan.

    Google Scholar 

  • Kingstone, A. (1992). Combining expectancies. The Quarterly Journal of Experimental Psychology, 44A, 69–104.

    Article  Google Scholar 

  • Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for studying human cognition. British Journal of Psychology, 99(3), 317–340.

    Article  PubMed  Google Scholar 

  • Krassner, M., Patera, W., Bulling, A. (2014) Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction. Computer Vision and Pattern Recognition. arXiv:1405.0006.

  • Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22(1), 79–86. https://doi.org/10.1214/aoms/1177729694.

    Article  Google Scholar 

  • Le Meur, O., & Baccino, T. (2013). Methods for comparing scanpaths and saliency maps: Strengths and weaknesses. Behavior Research Methods, 45, 251–266.

    Article  PubMed  Google Scholar 

  • Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10, 707–710.

    Google Scholar 

  • Levy, J., Foulsham, T., & Kingstone, A. (2012). Monsters are people too. Biology letters, Feb 23;9(1):20120850. https://doi.org/10.1098/rsbl.2012.0850.

    Article  PubMed  Google Scholar 

  • Mannan, S., Ruddock, K. H., & Wooding, D. S. (1995). Automatic control of saccadic eye movements made in visual inspection of briefly presented 2-D images. Spatial Vision, 9, 363–386. https://doi.org/10.1163/156856895X00052.

    Article  PubMed  Google Scholar 

  • Marwan, N., & Kurths, J. (2002). Nonlinear analysis of bivariate data with cross recurrence plots. Physics Letters A, 302, 299–307.

    Article  Google Scholar 

  • Mathôt, S., Cristino, F., Gilchrist, I. D., & Theeuwes, J. (2012). A simple way to estimate similarity between pairs of eye movement sequences. Journal of Eye Movement Research, 5(1).

    Google Scholar 

  • Needleman, S. B., & Wunsch, C. D. (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48(3), 443–453.

    Article  PubMed  Google Scholar 

  • Noton, D., & Stark, L. (1971). Scanpaths in eye movements during pattern perception. Science, 171, 308–311. https://doi.org/10.1126/science.171.3968.308.

    Article  PubMed  Google Scholar 

  • Nyström, M., & Holmqvist, K. (2008). Semantic override of low-level features in image viewing - both initially and overall. Journal of Eye Movement Research, 2, 1–11.

    Google Scholar 

  • Olk, B., & Kingstone, A. (2003). Why are antisaccades slower than prosaccades? A novel finding using a new paradigm. NeuroReport, 14, 151–155.

    Article  PubMed  Google Scholar 

  • Pellecchia, G. L., & Shockley, K. (2005). Cross recurrence quantification of interpersonal postural activity. In M. A. Riley & G. C. Van Orden (Eds.), Tutorials in contemporary nonlinear methods for the behavioral sciences (pp. 95–141). Washington, DC: National Science Foundation, Directorate for Social, Behavioral & Economic Sciences. Retrieved August 25, 2011, from www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp.

  • Posner, M. I. (1978). Chronometric explorations of mind. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Richardson, D. C., & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29, 1045–1060.

    Article  PubMed  Google Scholar 

  • Richardson, D. C., Dale, R., & Tomlinson, J. M. (2009). Conversation, gaze coordination, and beliefs about visual context. Cognitive Science, 33, 1468–1482. https://doi.org/10.1111/j.1551-6709.2009.01057.x.

    Article  PubMed  Google Scholar 

  • Riley, M. A., & Clark, S. (2003). Recurrence analysis of human postural sway during the sensory organization test. Neuroscience Letters, 342, 45–48.

    Article  PubMed  Google Scholar 

  • Risko, E. F., & Kingstone, A. (2011). Eyes wide shut: implied social presence, eye tracking and attention. Attention, Perception, & Psychophysics, 73(2), 291–296.

    Article  Google Scholar 

  • Shepherd, S. V., Steckenfinger, S. A., Hasson, U., & Ghasanfar, A. A. (2010). Human–monkey gaze correlations reveal convergent and divergent patterns of movie viewing. Current Biology, 20, 649–656.

    Article  PubMed  Google Scholar 

  • Shockley, K., Baker, A. A., Richardson, M. J., & Fowler, C. A. (2007). Articulatory constraints on interpersonal postural coordination. Journal of Experimental Psychology: Human Perception and Performance, 33(1), 201.

    PubMed  Google Scholar 

  • Shockley, K., Richardson, D. C., & Dale, R. (2009). Conversation and coordinative structures. Topics in Cognitive Science, 1, 305–319.

    Google Scholar 

  • Shockley, K., Santana, M.-V., & Fowler, C. A. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 326–332. https://doi.org/10.1037/0096-1523.29.2.326.

    Article  PubMed  Google Scholar 

  • Shockley, K., & Turvey, M. T. (2005). Encoding and retrieval during bimanual rhythmic coordination. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31(5), 980.

    Article  PubMed  Google Scholar 

  • Soto-Faraco, S., Morein-Zamir, S., & Kingstone, A. (2005). On audiovisual spatial synergy: The fragility of the phenomenon. Perception and Psychophysics, 67(3), 444–457.

    Article  PubMed  Google Scholar 

  • Stark, L. W. & Ellis, S. R. (1981) Scanpaths revisited: cognitive models direct active looking. In D. F. Fisher et al., (Eds.) Eye movements: cognition and visual perception, (pp. 193–226). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Tatler, B. W. (2007). The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of vision, 7(14), 4–4.

    Article  PubMed  Google Scholar 

  • Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005) Visual correlates of fixation selection: effects of scale and time. Vision Research, 45(5), 643–659

    Google Scholar 

  • Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006). Contextual guidance of eye movements and attention in real-worldscenes: The role of global features in object search. Psychological Review, 113(4), 766–786.

    Article  PubMed  Google Scholar 

  • Underwood, G., Foulsham, T., & Humphrey, K. (2009). Saliency and scan patterns in the inspection of real-world scenes: Eye movements during encoding and recognition. Visual Cognition, 17, 812–834.

    Article  Google Scholar 

  • Vig, E., Dorr, M., & Cox, D. (2014). Large-scale optimization of hierarchical features for saliency prediction in natural images. In The IEEE Conference on Pattern Recognition, Columbus Ohio, June 24–27.

    Google Scholar 

  • von der Malsburg, T., & Vasishth, S. (2011). What is the scanpath signature of syntactic reanalysis? Journal of Memory and Language, 65(2), 109–127.

    Article  Google Scholar 

  • Wagner, R. A., & Fischer, M. J. (1974). The string-to-string correction problem. Journal of the Association for Computing Machinery, 21, 168–178.

    Article  Google Scholar 

  • Walther, D., & Koch, C. (2006). Modeling attention to salient proto-objects. Neural Networks, 19, 1395–1407.

    Google Scholar 

  • Webber, C. L., Jr., & Zbilut, P. J. (2005). Recurrence quantification analysis of nonlinear dynamical systems. In M. A. Riley & G. C. Van Orden (Eds.), Tutorials in contemporary nonlinear methods for the behavioral sciences (pp. 26–94). Washington, DC: National Science Foundation, Directorate for Social, Behavioral & Economic Sciences. Retrieved August 25, 2011, from www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp.

  • West, J. M., Haake, A. R., Rozanski, E. P., & Karn, K. S. (2006). eyePatterns: software for identifying patterns and similarities across fixation sequences. Paper presented at the Proceedings of the 2006 Symposium on Eye Tracking Research & Applications, (pp. 149–154). San Diego, California, March 27–29, 2006.

    Google Scholar 

  • Woodford, O. (2014). SC - powerful image rendering. https://www.mathworks.com/matlabcentral/fileexchange/16233-sc-powerful-image-rendering. MATLAB Central File Exchange.

  • Yarbus, A. (1967). Eye movements and vision. New York, NY: Plenum Press.

    Book  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the support of the Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. Department of Psychology, University of British Columbia, Vancouver, BC, Canada

    Walter F. Bischof, Nicola C. Anderson & Alan Kingstone

Authors
  1. Walter F. Bischof
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicola C. Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Kingstone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walter F. Bischof .

Editor information

Editors and Affiliations

  1. Department of Child and Adolescent Psychiatry, University of Freiburg, Freiburg, Germany

    Christoph Klein

  2. Department of Psychology, University of Bonn, Bonn, Nordrhein-Westfalen, Germany

    Ulrich Ettinger

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bischof, W.F., Anderson, N.C., Kingstone, A. (2019). Temporal Methods for Eye Movement Analysis. In: Klein, C., Ettinger, U. (eds) Eye Movement Research. Studies in Neuroscience, Psychology and Behavioral Economics. Springer, Cham. https://doi.org/10.1007/978-3-030-20085-5_10

Download citation

Publish with us

Policies and ethics

Search

Navigation