The VERP Explorer: A Tool for Exploring Eye Movements of Visual-Cognitive Tasks Using Recurrence Plots

  • Çağatay DemiralpEmail author
  • Jesse Cirimele
  • Jeffrey Heer
  • Stuart K. Card
Conference paper
Part of the Mathematics and Visualization book series (MATHVISUAL)


Eye movement based analysis is becoming ever prevalent across domains with the commoditization of eye-tracking hardware. Eye-tracking datasets are, however, often complex and difficult to interpret and map to higher-level visual-cognitive behavior. Practitioners using eye tracking need tools to explore, characterize and quantify patterned structures in eye movements. In this paper, we introduce the VERP (Visualization of Eye movements with Recurrence Plots) Explorer, an interactive visual analysis tool for exploring eye movements during visual-cognitive tasks. The VERP Explorer couples conventional visualizations of eye movements with recurrence plots that reveal patterns of revisitation over time. We apply the VERP Explorer to the domain of medical checklist design , analyzing eye movements of doctors searching for information in checklists under time pressure.


Delaunay Triangulation Recurrence Plot Recurrence Quantification Analysis Alpha Shape Fixation Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Anderson, N.C., Bischof, W., Laidlaw, K., Risko, E., Kingstone, A.: Recurrence quantification analysis of eye movements. Behav. Res. Methods 45 (3):842–856 (2013)CrossRefGoogle Scholar
  2. 2.
  3. 3.
    Arriaga, A.F., Bader, A.M., et al.: A simulation-based trial of surgical-crisis checklists. New Engl. J. Med. 368 (15):1459–1460 (2013)Google Scholar
  4. 4.
    Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl, T.: State-of-the-art of visualization for eye tracking data. In: EuroVis – STARs. The Eurographics Association, Swansea (2014)Google Scholar
  5. 5.
    Boorman, D.: Safety benefits of electronic checklists – an analysis of commercial transport accidents. In: Proceedings of the 11th International Symposium on Aviation Psychology, Columbus (2001)Google Scholar
  6. 6.
    Bostock, M., Ogievetsky, V., Heer, J.: D3: data-driven documents. IEEE Trans. Visual. Comput. Graph. 17 (12):2301–2309 (2011)CrossRefGoogle Scholar
  7. 7.
    Burch, M., Kull, A., Weiskopf, D.: AOI rivers for visualizing dynamic eye gaze frequencies. Comput. Graph. Forum 32:281–290 (2013)CrossRefGoogle Scholar
  8. 8.
    Burch, M., Schmauder, H., Raschke, M., Weiskopf, D.: Saccade plots. In: Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’14). ACM, New York (2014)Google Scholar
  9. 9.
    Burian, B.K., Barshi, I., Dismukes, K.: The challenge of aviation emergency and abnormal situations. Technical report, NASA Ames Research Center (2005)Google Scholar
  10. 10.
    Cirimele, J., Wu, L., Leach, K., Card, S.K., Harrison, T.K., Chu, L., Klemmer, S., RapidRead: step-at-a-glance crisis checklists. In: Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg (2014)Google Scholar
  11. 11.
    Dale, R., Warlaumont, A.S., Richardson, D.C.: Nominal cross recurrence as a generalized lag sequential analysis for behavioral streams. Int. J. Bifurcation and Chaos 21 (04):1153–1161 (2011)Google Scholar
  12. 12.
    Degani, A., Wiener, E.L.: Human factors of flight-deck checklists: the normal checklist. Technical report, NASA Ames Research Center (1990)Google Scholar
  13. 13.
    Demiralp, Ç., Bernstein, M., Heer, J.: Learning perceptual kernels for visualization design. IEEE Trans. Visual. Comput. Graph. (Proc. InfoVis) 20 (12), 1933–1942 (2014)Google Scholar
  14. 14.
    Eckmann, J.-P., Kamphorst, S.O., Ruelle, D.: Recurrence plots of dynamical systems. EPL (Eur. Lett.) 4 (9), 973 (1987)Google Scholar
  15. 15.
    Edelsbrunner, H., Mcke, E.P.: Three-dimensional alpha shapes. ACM Trans. Graph. 13 (1), 43–72 (1994)CrossRefzbMATHGoogle Scholar
  16. 16.
    Gawande, A.: The Checklist Manifesto: How to Get Things Right. Metropolitan Books, New York (2009)Google Scholar
  17. 17.
    Harrison, T.K., Manser, T., Howard, S.K., Gaba, D.M.: Use of cognitive aids in a simulated anesthetic crisis. Anesthe. Analge. 103 (3), 551–556 (2006)CrossRefGoogle Scholar
  18. 18.
    Harrower, M., Brewer, C.A.: an online tool for selecting colour schemes for maps. Cartogr. J. 40 (1), 27–37 (2003)Google Scholar
  19. 19.
    Havre, S., Hetzler, B., Nowell L.: ThemeRiver: visualizing theme changes over time. In: IEEE Proceedings of InfoVis’00, Salt Lake City (2000)Google Scholar
  20. 20.
    Haynes, A., Weiser, T., Berry, W., Lipsitz, S., Breizat, A.-H., Dellinger, E., Herbosa, T., Joseph, S., Kibatala, P., Lapitan, M., Merry, A., Moorthy, K., Reznick, R., Taylor, B., Gawande, A.: A surgical safety checklist to reduce morbidity and mortality in a global population. New Eng. J. Med. 360, 491–499 (2009)CrossRefGoogle Scholar
  21. 21.
  22. 22.
    James, J.T.: A new, evidence-based estimate of patient harms associated with hospital care. J. Patient Saf. 9 (3), 122–128 (2013)CrossRefGoogle Scholar
  23. 23.
    Kieras, D.E., Hornof, A.J.: Towards accurate and practical predictive models of active-vision-based visual search. In: Proceedings of CHI’14, pp. 3875–3884. ACM, New York (2014)Google Scholar
  24. 24.
    Kurzhals, K., Weiskopf, D.: Space-time visual analytics of eye-tracking data for dynamic stimuli. IEEE Trans. Visual. Comput. Graph. 19 (12), 2129–2138 (2013)CrossRefGoogle Scholar
  25. 25.
    Li, X., Çöltekin, A., Kraak, M.-J.: Visual exploration of eye movement data using the space-time-cube. In: Fabrikant, S., Reichenbacher, T., van Kreveld, M., Schlieder, C. (eds) Geographic Information Science, vol. 6292 of LNCS, pp. 295–309. Springer, Berlin/Heidelberg (2010)Google Scholar
  26. 26.
    Makary, M.A., Holzmueller, C.G., Thompson, D., Rowen, L., Heitmiller, E.S., Maley, W.R., Black, J.H., Stegner, K., Freischlag, J.A., Ulatowski, J.A., Pronovost, P.J. Operating room briefings: working on the same page. Jt. Comm. J. Qual. Patient Saf./Jt. Comm. Res. 32 (6), 351355 (2006)Google Scholar
  27. 27.
    Marwan, N.: A historical review of recurrence plots. Eur. Phys. J. Spec. Top. 164 (1), 3–12 (2008)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Marwan, N., Romano, M.C., Thiel, M., Kurths, J.: Recurrence plots for the analysis of complex systems. Phys. Rep. 438 (56), 237–329 (2007)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., Sexton, B., Hyzy, R., Welsh, R., Roth, G., Bander, J., Kepros, J., Goeschel, C.: An intervention to decrease catheter-related bloodstream infections in the ICU. New Eng. J. Med. 355 (26), 2725–2732 (2006)CrossRefGoogle Scholar
  30. 30.
    Raschke, M., Chen, X., Ertl, T.: Parallel scan-path visualization. In: Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’12). ACM, Austin (2012)Google Scholar
  31. 31.
    Richardson, D.C., Dale, R., Kirkham, N.Z.: The art of conversation is coordination: common ground and the coupling of eye movements during dialogue. Psychol. Sci. 18 (5), 407–413 (2007)CrossRefGoogle Scholar
  32. 32.
    Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of ETRA’00. ACM, Palm Beach Gardens (2000)Google Scholar
  33. 33.
    Tsang, H.Y., Tory, M., Swindells, C.: eSeeTrack–visualizing sequential fixation patterns. IEEE Trans. Visual. Comput. Graph. 16 (6), 953–962 (2010)CrossRefGoogle Scholar
  34. 34.
    Verdaasdonk, E.G.G., Stassen, L.P.S., Widhiasmara, P.P., Dankelman, J.: Requirements for the design and implementation of checklists for surgical processes. Surg. Endosc. 23 (4), 715–726 (2008)CrossRefGoogle Scholar
  35. 35.
    Wattenberg, M., Viegas, F.: The word tree, an interactive visual concordance. IEEE Trans. Visual. Comput. Graph. 14 (6), 1221–1228 (2008)CrossRefGoogle Scholar
  36. 36.
    Winters, B.D., Gurses, A.P., Lehmann, H., Sexton, J.B., Rampersad, C., Pronovost, P.J.: Clinical review: checklists – translating evidence into practice. Crit. Care 13 (6), 210 (2009)CrossRefGoogle Scholar
  37. 37.
    Wu, D.W.-L., Anderson, N.C., Bischof, W.F., Kingstone, A.: Temporal dynamics of eye movements are related to differences in scene complexity and clutter. J. Vis. 14 (9), 8 (2014)CrossRefGoogle Scholar
  38. 38.
    Ziewacz, J.E., Arriaga, A.F., Bader, A.M., Berry, W.R., Edmondson, L., Wong, J.M., Lipsitz, S.R., Hepner, D.L., Peyre, S., Nelson, S., Boorman, D.J., Smink, D.S., Ashley, S.W., Gawande, A.A.: Crisis checklists for the operating room: development and pilot testing. J. Am. Coll. Surg. 213 (2), 212–217 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Çağatay Demiralp
    • 1
    Email author
  • Jesse Cirimele
    • 2
  • Jeffrey Heer
    • 3
  • Stuart K. Card
    • 4
  1. 1.IBM ResearchYorktown HeightsUSA
  2. 2.Tangible Play, Inc.Palo AltoUSA
  3. 3.University of WashingtonSeattleUSA
  4. 4.Stanford UniversityStanfordUSA

Personalised recommendations