Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Eye Tracking in Virtual Reality

  • Mehmet Ilker BerkmanEmail author
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_170-1



Eye tracking techniques have many applications of research in medicine, psychology, marketing, and human factors. It is also used as a human–computer interface for applications such as gaze-based typing and communication (Majaranta 2012; Ten Kate et al. 1979), driving safety (Chen et al. 2018; Grace et al. 1998; Kutila et al. 2007; Sinha et al. 2018), and gaming (Smith and Graham 2006; Tobii Gaming n.d.). Besides being a research tool and human–computer interface in VR (virtual reality), gaze-based techniques are also used to enhance the graphics quality and performance of displays with methods of gaze prioritized graphics, also known as foveated rendering. Furthermore, statistical models of eye tracking data are employed to provide eye movements for computer-generated avatars (Gemmell et al. 2000; Seele et al. 2017; Vinayagamoorthy et al. 2004).

The human sight is limited to...

This is a preview of subscription content, log in to check access.


  1. Advani, S., Sustersic, J., Irick, K., Narayanan, V.: A multi-resolution saliency framework to drive foveation. In: 2013 IEEE. International Conference on Acoustics, Speech and Signal Processing (2013)Google Scholar
  2. Albert, R., Patney, A., Luebke, D., Kim, J.: Latency Requirements for Foveated Rendering in Virtual Reality. ACM Transactions on Applied Perception. 14, 1–13 (2017)CrossRefGoogle Scholar
  3. Arabadzhiyska, E., Tursun, O., Myszkowski, K., Seidel, H., Didyk, P.: Saccade landing position prediction for gaze-contingent rendering. ACM Transactions on Graphics. 36, pp. 1–12 (2017)CrossRefGoogle Scholar
  4. Arndt, S., Antons, J.N.: Enhancing video streaming using real-time gaze tracking. In: Proceedings of the 5th ISCA/DEGA Workshop on Perceptual Quality of Systems, pp. 6–9 (2016)Google Scholar
  5. Baldauf, M., Fröhlich, P., Hutter, S.: KIBITZER: a wearable system for eye-gaze-based mobile urban exploration. In: Proceedings of the 1st Augmented Human International Conference, pp. 9–13 (2010)Google Scholar
  6. Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl, T.: Visualization of eye tracking data: a taxonomy and survey. Comput. Graph. Forum. 36, 260–284 (2017)CrossRefGoogle Scholar
  7. Carnegie, K., Rhee, T.: Reducing visual discomfort with HMDs using dynamic depth of field. IEEE Comput. Graph. Appl. 35, 34–41 (2015)CrossRefGoogle Scholar
  8. Chen, L.B., Chang, W.J., Hu, W.W., Wang, C.K., Lee, D.H., Chiou, Y.Z.: A band-pass IR light photodetector for wearable intelligent glasses in a drowsiness-fatigue-detection system. In: Consumer Electronics (ICCE), 2018 IEEE International Conference on, pp. 1–2. IEEE (2018)Google Scholar
  9. Duchowski, A.T.: Eye Tracking Methodology. Theory and Practice. Springer International Publishing AG, Cham (2017)CrossRefGoogle Scholar
  10. Duchowski, A.T.: Gaze-based interaction: a 30 year retrospective. Comput. Graph. 73, 59–69 (2018)CrossRefGoogle Scholar
  11. Duchowski, A.T., Jörg, S.: Eye animation. In: Müller, B., Wolf, S.I. (eds.) Handbook of Human Motion, Springer Nature, Cham, pp. 1–19 (2016)Google Scholar
  12. Duchowski, A.T., Shivashankaraiah, V., Rawls, T., Gramopadhye, A.K., Melloy, B.J., Kanki, B.: Binocular eye tracking in virtual reality for inspection training. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, pp. 89–96 (2000)Google Scholar
  13. Duchowski, A.T., Medlin, E., Cournia, N., Gramopadhye, A., Melloy, B., Nair, S.: 3D eye movement analysis for VR visual inspection training. In: Proceedings of the Symposium on Eye Tracking Research & Applications – ETRA 02 (2002)Google Scholar
  14. Duchowski, A.T., Cournia, N., Cumming, B., Mccallum, D., Gramopadhye, A., Greenstein, J., Sadasivan, S., Tyrrell, R.A.: Visual deictic reference in a collaborative virtual environment. In: Proceedings of the Eye Tracking Research & Applications Symposium on Eye Tracking Research & Applications – ETRA2004 (2004)Google Scholar
  15. Duchowski, A.T., House, D.H., Gestring, J., Wang, R.I., Krejtz, K., Krejtz, I., Mantiuk, R., Bazyluk, B.: Reducing visual discomfort of 3D stereoscopic displays with gaze-contingent depth-of-field. In: Proceedings of the ACM Symposium on Applied Perception – SAP 14 (2014)Google Scholar
  16. Gemmell, J., Toyama, K., Zitnick, C.L., Kang, T., Seitz, S.: Gaze awareness for video-conferencing: a software approach. IEEE MultiMedia. 7(4), 26–35 (2000)CrossRefGoogle Scholar
  17. Grace, R., Byrne, V., Bierman, D., Legrand, J.-M., Gricourt, D., Davis, B., Staszewski, J., Carnahan, B.: A drowsy driver detection system for heavy vehicles. In: 17th DASC. AIAA/IEEE/SAE. Digital Avionics Systems Conference (1998)Google Scholar
  18. Greenwald, S.W., Loreti, L., Funk, M., Zilberman, R., Maes, P.: Eye gaze tracking with google cardboard using purkinje images. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, pp. 19–22 (2016)Google Scholar
  19. Haffegee, A., Barrow, R.: Eye tracking and gaze based interaction within immersive virtual environments. In: International Conference on Computational Science, pp. 729–736. Springer, Berlin/Heidelberg (2009)Google Scholar
  20. Hollomon, M.J., Kratchounova, D., Newton, D.C., Gildea, K., Knecht, W.R.: Current status of gaze control research and technology literature. Technical report. Federal Aviation Administration, Washington, DC (2017)Google Scholar
  21. Itoh, K., Hansen, J.P., Nielsen, F.R.: Cognitive modelling of a ship navigator based on protocol and eye-movement analysis. Trav. Hum. 61, 99–127 (1998)Google Scholar
  22. Itoh, K., Tanaka, H., Seki, M.: Eye-movement analysis of track monitoring patterns of night train operators: effects of geographic knowledge and fatigue. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 44(27), 360–363 (2000)CrossRefGoogle Scholar
  23. Iwamoto, K., Katsumata, S., Tanie, K.: An eye movement tracking type head mounted display for virtual reality system: evaluation experiments of a prototype system. In: IEEE International Conference on Humans, Information and Technology, no. 1, pp. 13–18 (1994)Google Scholar
  24. Jacob, R.J.: Eye tracking in advanced interface design. In: Barfield, W., Furness, T.A. (eds.) Virtual Environments and Advanced Interface Design, pp. 258–288. Oxford University Press, New York (1995)Google Scholar
  25. Just, M.A., Carpenter, P.A.: Eye fixations and cognitive processes. Cogn. Psychol. 8, 441–480 (1976)CrossRefGoogle Scholar
  26. Kellnhofer, P., Didyk, P., Myszkowski, K., Hefeeda, M.M., Seidel, H.-P., Matusik, W.: GazeStereo3D: seamless disparity manipulations. ACM Trans. Graph. 35, 1–13 (2016)Google Scholar
  27. Khamis, M., Oechsner, C., Alt, F., Bulling, A.: VRpursuits: interaction in virtual reality using smooth pursuit eye movement. Proceedings of the 2018 International Conference on Advanced Visual Interfaces - AVI ’18 May 29-June 1, Castiglione della Pescaia, Italy (2018)Google Scholar
  28. Koulieris, G., Drettakis, G., Cunningham, D., Mania, K.: An automated high-level saliency predictor for smart game balancing. ACM Trans. Appl. Percept. 11, 1–21 (2015)CrossRefGoogle Scholar
  29. Koulieris, G., Drettakis, G., Cunningham, D., Mania, K.: Gaze prediction using machine learning for dynamic stereo manipulation in games. In: 2016 IEEE Virtual Reality (VR) (2016)Google Scholar
  30. Kulshreshth, A., Laviola, J.J.: Dynamic stereoscopic 3D parameter adjustment for enhanced depth discrimination. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems – CHI 16 (2016)Google Scholar
  31. Kutila, M., Jokela, M., Markkula, G., Rue, M.R.: Driver distraction detection with a camera vision system. In: 2007 IEEE International Conference on Image Processing (2007)Google Scholar
  32. Lavoué, G., Cordier, F., Seo, H., Larabi, M.: Visual attention for rendered 3D shapes. Comput. Graph. Forum. 37, 191–203 (2018)CrossRefGoogle Scholar
  33. Levoy, M., Whitaker, R.: Gaze-directed volume rendering. ACM SIGGRAPH Comput. Graph. 24, 217–223 (1990)CrossRefGoogle Scholar
  34. Luebke, D., Erikson, C.: View-dependent simplification of arbitrary polygonal environments. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques – SIGGRAPH ‘97 (1997)Google Scholar
  35. Luebke, D., Hallen, B.: Perceptually Driven Simplification for Interactive Rendering. Eurographics. 223–234 (2001)Google Scholar
  36. Majaranta, P.: Communication and text entry by gaze. In: Majaranta, P., et al. (eds.) Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies, pp. 63–77. IGI Global, Hershey (2012)CrossRefGoogle Scholar
  37. Mine, M.: Virtual Environment Interaction Techniques. UNC Chapel Hill Computer Science technical report TR95–018. University of North Carolina, Chapel Hill (1995)Google Scholar
  38. Mon-Williams, M., Wann, J.P.: Binocular virtual reality displays: when problems do and don’t occur. Hum. Factors. 40, 42–49 (1998)CrossRefGoogle Scholar
  39. Murphy, H., Duchowski, A.T.: Gaze-contingent level of detail rendering. In: EuroGraphics (2001)Google Scholar
  40. Murphy, H., Duchowski, A., Tyrrell, R.: Hybrid image/model-based gaze-contingent rendering. ACM Trans. Appl. Percept. 5, 1–21 (2009)CrossRefGoogle Scholar
  41. Murray, N., Roberts, D., Steed, A., Sharkey, P., Dickerson, P., Rae, J., Wolff, R.: Eye gaze in virtual environments: evaluating the need and initial work on implementation. Concurr. Comput. Pract. Exp. 21(11), 1437–1449 (2009)CrossRefGoogle Scholar
  42. Ohshima, T., Yamamoto, H., Tamura, H.: Gaze-directed adaptive rendering for interacting with virtual space. In: Proceedings of the IEEE Virtual Reality Annual International Symposium, pp. 103–110 (1996)Google Scholar
  43. Paletta, L., Santner, K., Fritz, G., Mayer, H., Schrammel, J.: 3D attention: measurement of visual saliency using eye tracking glasses. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems, pp. 199–204. ACM (2013)Google Scholar
  44. Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., Lefohn, A.: Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph. 35, 1–12 (2016)CrossRefGoogle Scholar
  45. Pfeiffer, T.: Towards gaze interaction in immersive virtual reality: evaluation of a monocular eye tracking set-up. In: Virtuelle und Erweiterte Realität-Fünfter Workshop der GI-Fachgruppe VR/AR (2008)Google Scholar
  46. Pfeiffer, T.: Measuring and visualizing attention in space with 3D attention volumes. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 29–36 (2012)Google Scholar
  47. Pfeiffer, T., Memili, C.: Model-based real-time visualization of realistic three-dimensional heat maps for mobile eye tracking and eye tracking in virtual reality. In: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, pp. 95–102 (2016)Google Scholar
  48. Pfeiffer, T., Latoschik, M.E., Wachsmuth, I.: Evaluation of binocular eye trackers and algorithms for 3D gaze interaction in virtual reality environments. J. Virtual Real. Broadcast. 5(16), 1660 (2008)Google Scholar
  49. Pfeuffer, K., Mayer, B., Mardanbegi, D., Gellersen, H.: Gaze + pinch interaction in virtual reality. In: Proceedings of the 5th Symposium on Spatial User Interaction, pp. 99–108 (2017)Google Scholar
  50. Piumsomboon, T., Lee, G., Lindeman, R., Billinghurst, M.: Exploring natural eye-gaze-based interaction for immersive virtual reality. 2017 IEEE Symposium on 3D User Interfaces (3DUI). 18–19 March, Los Angeles, CA, USA. pp. 36–39 (2017)Google Scholar
  51. Pohl, D., Zhang, X., Bulling, A.: Combining eye tracking with optimizations for lens astigmatism in modern wide-angle HMDs. In: 2016 IEEE Virtual Reality (VR) (2016)Google Scholar
  52. Poole, A., Ball, L.J.: Eye tracking in HCI and usability research. In: Encyclopedia of human computer interaction, vol. 1, Idea Group Reference, London, UK pp. 211–219 (2006)Google Scholar
  53. Ramloll, R., Trepagnier, C., Sebrechts, M., Beedasy, J.: Gaze data visualization tools: opportunities and challenges. In: Proceedings of the 8th International Conference on Information Visualisation, pp. 173–180 (2004)Google Scholar
  54. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372–422 (1998)CrossRefGoogle Scholar
  55. Rayner, K.: The 35th Sir Frederick Bartlett lecture: eye movements and attention in reading, scene perception, and visual search. Q. J. Exp. Psychol. 62(8), 1457–1506 (2009)CrossRefGoogle Scholar
  56. Roberts, D.J., Fairchild, A.J., Campion, S.P., Ohare, J., Moore, C.M., Aspin, R., Duckworth, T., Gasparello, P., Tecchia, F.: withyou – an experimental end-to-end telepresence system using video-based reconstruction. IEEE J. Sel. Top. Signal Process. 9, 562–574 (2015)CrossRefGoogle Scholar
  57. Roth, T., Weier, M., Hinkenjann, A., Li, Y., Slusallek, P.: An analysis of eye-tracking data in foveated ray tracing. In: 2016 IEEE Second Workshop on Eye Tracking and Visualization (ETVIS) (2016)Google Scholar
  58. Roth, T., Weier, M., Hinkenjann, A., Li, Y., Slusallek, P.: A quality-centered analysis of eye tracking data in foveated rendering. J. Eye Mov. Res. 10(5) pp. 1–12 (2017)Google Scholar
  59. Rötting, M., Göbel, M., Springer, J.: Automatic object identification and analysis of eye movement record-ings. MMI Interakt. 1(2) pp. 1–7 (1999)Google Scholar
  60. Sadasivan, S., Rele, R., Greenstein, J.S., Duchowski, A.T., Gramopadhye, A.K.: Simulating on-the-job training using a collaborative virtual environment with head slaved visual deictic reference. In: Proceedings of HCI International Annual Conference, pp. 22–27 (2005)Google Scholar
  61. Schulz, C.M., Schneider, E., Fritz, L., Vockeroth, J., Hapfelmeier, A., Brandt, T., Kochs, E.F., Schneider, G.: Visual attention of anaesthetists during simulated critical incidents. Br. J. Anaesth. 106(6), 807–813 (2011)CrossRefGoogle Scholar
  62. Seele, S., Misztal, S., Buhler, H., Herpers, R., Schild, J.: Here’s looking at you anyway!: how important is realistic gaze behavior in co-located social virtual reality games? In: Proceedings of the Annual Symposium on Computer-Human Interaction in Play, pp. 531–540. ACM (2017)Google Scholar
  63. Sinha, O., Singh, S., Mitra, A., Ghosh, S.K., Raha, S.: Development of a drowsy driver detection system based on EEG and IR-based eye blink detection analysis. In: Bera, R., Kumar, S., Chakraborty, S.S. (eds.) Advances in Communication, Devices and Networking, Springer Nature Pte Ltd., Singapore pp. 313–319 (2018)CrossRefGoogle Scholar
  64. Smith, J.D., Graham, T.C.: Use of eye movements for video game control. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (2006)Google Scholar
  65. Stellmach, S., Nacke, L., Dachselt, R.: 3D attentional maps: aggregated gaze visualizations in three-dimensional virtual environments. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 345–348. ACM (2010)Google Scholar
  66. Steptoe, W., Oyekoya, O., Murgia, A., Wolff, R., Rae, J., Guimaraes, E., Roberts, D., Steed, A.: Eye tracking for avatar eye gaze control during object-focused multiparty interaction in immersive collaborative virtual environments. In: 2009 IEEE Virtual Reality Conference (2009)Google Scholar
  67. Swafford, N., Iglesias-Guitian, J., Koniaris, C., Moon, B., Cosker, D., Mitchell, K.: User, metric, and computational evaluation of foveated rendering methods. In: Proceedings of the ACM Symposium on Applied Perception – SAP ‘16 (2016)Google Scholar
  68. Tanriverdi, V., Jacob, R.J.: Interacting with eye movements in virtual environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 265–272. ACM (2000)Google Scholar
  69. Ten Kate, J.H., Frietman, E.E., Willems, W., Romeny, B.T.H., Tenkink, E.: Eye-switch controlled communication aids. In: Proceedings of the 12th International Conference on Medical & Biological Engineering, pp. 19–20 (1979)Google Scholar
  70. Triesch, J., Sullivan, B.T., Hayhoe, M.M., Ballard, D.H.: Saccade contingent updating in virtual reality. In: Proceedings of the Symposium on Eye Tracking Research & Applications – ETRA 02 (2002)Google Scholar
  71. Tsang, H.Y., Tory, M., Swindells, C.: eSeeTrack – visualizing sequential fixation patterns. IEEE Trans. Vis. Comput. Graph. 16(6), 953–962 (2010)CrossRefGoogle Scholar
  72. Vinayagamoorthy, V., Garau, M., Steed, A., Slater, M.: An eye gaze model for dyadic interaction in an immersive virtual environment: practice and experience. Comput. Graph. Forum. 23(1), 1–11 (2004)CrossRefGoogle Scholar
  73. Watson, B., Walker, N., Hodges, L., Worden, A.: Managing level of detail through peripheral degradation: effects on search performance with a head-mounted display. ACM Trans. Comput.-Hum. Interact. 4, 323–346 (1997)CrossRefGoogle Scholar
  74. Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E.: Let’s look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 107–114. ACM (2012)Google Scholar
  75. Weier, M., Roth, T., Kruijff, E., Hinkenjann, A., Pérard-Gayot, A., Slusallek, P., Li, Y.: Foveated real-time ray tracing for head-mounted displays. Comput. Graph. Forum. 35, 289–298 (2016)CrossRefGoogle Scholar
  76. Zeleznik, R.C., Forsberg, A.S., Schulze, J.P.: Look-that-there: exploiting gaze in virtual reality interactions. Technical report, Technical Report CS-05 (2005)Google Scholar
  77. Zha, H., Makimoto, Y., Hasegawa, T.: Dynamic gaze-controlled levels of detail of polygonal objects in 3-D environment modeling. In: Second International Conference on 3-D Digital Imaging and Modeling, pp. 321–330 (1999)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Communication DesignBahçeşehir University Faculty of CommunicationIstanbulTurkey