Scanpath comparisons for complex visual search in a naturalistic environment

  • Mary E. FrameEmail author
  • Rik Warren
  • Anna M. Maresca


Naturalistic surveillance tasks provide a rich source of eye-tracking data. It can be challenging to make meaningful comparisons using standard eye-tracking analysis techniques such as saccade frequency or blink rate in surveillance studies due to the temporal irregularity of events of interest. Naturalistic research environments present unique challenges, such as requiring specialized or expert analysts, small sample size, and long data collection sessions. These constraints demand rich data and sophisticated analyses, particularly in prescriptive naturalistic environments where problems must be thoroughly understood to implement effective and practical solutions. Using a small sample of expert surveillance analysts and an equal-sized sample of novices, we computed scanpath similarity on a variety of surveillance data using the ScanMatch Matlab tool. ScanMatch implements an algorithm initially developed for DNA protein sequence comparisons and provides a similarity score for two scanpaths based on their morphology and, optionally, duration in an area of interest. Both experts and novices showed equal dwell time on targets regardless of identification accuracy and both samples showed higher scanpath consistency across participants as a function of target type rather than individual subjects showing a particular scanpath preference. Our results show that scanpath analysis can be leveraged as a highly effective computer-based methodology to characterize surveillance identification errors and guide the implementation of solutions. Similarity scores can also provide insight into processes guiding visual search.


Visual search Scanpath analysis Applied research 



  1. Boydstun, A. S., Maresca, A. M., Saunders, E., & Stanfill, C. (2018) Real-time annotation and dissemination tool (RTAD) demo. Dayton: Air Force Research Laboratory.Google Scholar
  2. Brooke, J., et al. (1996). SUS A quick and dirty usability scale. Usability Evaluation in Industry, 189(194), 4–7.Google Scholar
  3. Cristino, F., Mathôt, S., Theeuwes, J., & Gilchrist, I. D. (2010). Scanmatch: a novel method for comparing fixation sequences. Behavior Research Methods, 42(3), 692–700.CrossRefPubMedGoogle Scholar
  4. Dewhurst, R., Nyström, M., Jarodzka, H., Foulsham, T., Johansson, R., & Holmqvist, K. (2012). It depends on how you look at it: Scanpath comparison in multiple dimensions with multimatch, a vector-based approach. Behavior Research Methods, 44(4), 1079–1100.CrossRefPubMedGoogle Scholar
  5. Drew, T., Võ, M. L. H., & Wolfe, J. M. (2013). The invisible gorilla strikes again: Sustained inattentionalblindness in expert observers. Psychological Science, 24(9), 1848–1853.CrossRefPubMedPubMedCentralGoogle Scholar
  6. Drew, T., & Stothart, C. (2016). Clarifying the role of targetsimilarity, task relevance and feature-based suppression during sustained inattentional blindness. Journal of Vision, 16(15), 1–9.CrossRefGoogle Scholar
  7. Drew, T., Boettcher, S. E., & Wolfe, J. M. (2017). One visual search, many memory searches: an eye-tracking investigationof hybrid search. Journal of Vision, 17(11), 1–10.CrossRefGoogle Scholar
  8. Foerster, R. M., & Schneider, W. X. (2013). Functionally sequenced scanpath similarity method (funcsim): Comparing and evaluating scanpath similarity based on a task’s inherent sequence of functional (action)units. Journal of Eye Movement Research, 6(5), 1–22.Google Scholar
  9. Folk, C. L., Remington, R. W., Johnston, J. C., & et al. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044.PubMedGoogle Scholar
  10. Foulsham, T., & Underwood, G. (2008). What can saliency modelspredict about eye movements? spatial and sequential aspects of fixations during encoding and recognition. Journal of Vision, 8(2), 6–6.CrossRefPubMedGoogle Scholar
  11. Gaspelin, N., Leonard, C. J., & Luck, S. J. (2017). Suppression of overt attentional capture by salient-but-irrelevant colorsingletons. Attention, Perception, and Psychophysics, 79(1), 45–62.CrossRefGoogle Scholar
  12. Hart, S. G. (2006). NASA-Task load index (NASA-TLX); 20 years later. Inproceedings of the human factors and ergonomics society annual meeting, 50, 904–908.CrossRefGoogle Scholar
  13. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (task load index): Results of empiricaland theoretical research. Advances in Psychology, 52, 139–183.CrossRefGoogle Scholar
  14. Jarodzka, H., Holmqvist, K., & Nyström, M. (2010). A vector-based, multidimensional scanpath similarity measure. In Proceedings of the 2010 symposium on eye-tracking research & applications (pp. 211–218).Google Scholar
  15. Kübler, T., Eivazi, S., & Kasneci, E. (2015). Automated visual scanpath analysis reveals the expertise level ofmicro-neurosurgeons. In Miccai workshop on interventional microscopy.Google Scholar
  16. Leber, A. B., & Egeth, H. E. (2006). It’s under control: Top-down search strategies can override attentionalcapture. Psychonomic Bulletin & Review, 13(1), 132–138.CrossRefGoogle Scholar
  17. Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8), 707–710.Google Scholar
  18. Poole, A., & Ball, L. J. (2006). Eye tracking in HCI and usability research. Encyclopedia of Human Computer Interaction, 1, 211–219.CrossRefGoogle Scholar
  19. Sawaki, R., & Luck, S. J. (2010). Capture versus suppression ofattention by salient singletons: Electrophysiological evidence for an automatic attend-to-me signal. Attention, Perception, and Psychophysics, 72(6), 1455–1470.CrossRefGoogle Scholar
  20. Siegle, G. J., Ichikawa, N., & Steinhauer, S. (2008). Blink before and after you think: blinksoccur prior to and following cognitive load indexed by pupillary responses. Psychophysiology, 45(5), 679–687.CrossRefPubMedGoogle Scholar
  21. Stern, J. A., Boyer, D., & Schroeder, D. (1994). Blink rate: a possible measure of fatigue. Human Factors, 36(2), 285–297.CrossRefPubMedGoogle Scholar
  22. Theeuwes, J., Kramer, A. F., Hahn, S., & Irwin, D. E. (1998). Our eyes do not always go where we want them to go: Capture of the eyesby new objects. Psychological Science, 9(5), 379–385.CrossRefGoogle Scholar
  23. Theeuwes, J., De Vries, G. J., & Godijn, R. (2003). Attentional and oculomotor capture with static singletons. Attention, Perception, and Psychophysics, 65(5), 735–746.CrossRefGoogle Scholar
  24. Van Orden, K. F., Limbert, W., Makeig, S., & Jung, T P. (2001). Eye activity correlates of workload during a visuospatial memory task. Human Factors, 43(1), 111–121.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. (This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply) 2018

Authors and Affiliations

  1. 1.Wright State UniversityDaytonUSA
  2. 2.Air Force Research LaboratoryWright-Patterson AFB OhioDaytonUSA

Personalised recommendations