Game Analytics pp 543-583 | Cite as

Visual Attention and Gaze Behavior in Games: An Object-Based Approach

  • Veronica Sundstedt
  • Matthias Bernhard
  • Efstathios Stavrakis
  • Erik Reinhard
  • Michael Wimmer


In the design of interactive applications, notably games, a recent trend is to understand player behavior by investigating telemetry logs as is the focus of many chapters in this book or by integrating the use of psychophysics as is the subject of Chaps. 26 and 27. In addition to these valuable methods, measuring, where players are likely to focus, could be a very useful tool in the arsenal of game designers. This knowledge can be utilized to help game designers decide how and where to allocate computing resources, such as rendering and various kinds of simulations of physical properties. This leaves as many computing cycles as possible free to carry out other tasks. Therefore, the perceived realism of a game can be increased by perceptually optimizing calculations that are computationally intensive, including physically based lighting (e.g. ray-tracing Cater et al. 2003), animations (e.g. crowds of characters McDonnell et al. 2009), physically correct simulations of the interaction of materials (e.g. collision detection (O’Sullivan 2005), natural behavior of clothes or fluids etc.). Level of Detail variants of simulation or rendering techniques can be used in regions which are less attended by the player, while accurate simulations can be used within the expected focus of a user. Verifying or improving game mechanics and AI could be other uses.



  1. Baylis, G. C., & Driver, J. (1993). Visual attention and objects: evidence for hierarchical coding of location. Journal of Experimental Psychology: Human Perception and Performance, 19(3), 451–470.CrossRefGoogle Scholar
  2. Behrmann, M., Zemel, R. S., & Mozer, M. C. (1998). Object-based attention and occlusion evidence from normal participants and a computational model. Journal of Experimental Psychology: Human Perception and Performance, 24, 1011–1036.CrossRefGoogle Scholar
  3. Bernhard, M., Stavrakis, E., & Wimmer, M. (2010). An empirical pipeline to derive gaze prediction heuristics for 3D action games. ACM Transactions on Applied Perception (TAP), 8(1), 4:1–4:30.Google Scholar
  4. Bernhard, M., Zhang, L., & Wimmer, M. (2011). Manipulating attention in computer games. IVMSP workshop, 2011 IEEE 10th (pp. 153–158), Ithaca, NY.Google Scholar
  5. Canosa, R. L., Pelz, J. B., Mennie, N. R., & Peak, J. (2003). High-level aspects of oculomotor control during viewing of natural-task images. In B. E. Rogowitz & T. N. Pappas (Eds.), Human vision and electronic imaging VIII. Proceedings of the SPIE in presented at the Society of Photo-Optical Instrumentation Engineers (SPIE) conference (pp. 240–251). Santa Clara, CA.Google Scholar
  6. Castiello, U., & Umiltà, C. (1990). Size of the attentional focus and efficiency of processing. Acta Psychologica, 73(3), 195–209.CrossRefGoogle Scholar
  7. Cater, K., Chalmers, A., & Ledda, P. (2002). Selective quality rendering by exploiting human inattentional blindness: Looking but not seeing. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (pp. 17–24). Hong Kong, China.Google Scholar
  8. Cater, K., Chalmers, A., & Ward, G. (2003). Detail to attention: Exploiting visual tasks for selective rendering. In Proceedings of the 14th Eurographics workshop on Rendering in EGRW ’03 (pp. 270–280). Aire-la-Ville, Switzerland: Eurographics Association.Google Scholar
  9. Chaney, I. M., Lin, K.-H., & Chaney, J. (2004). The effect of billboards within the gaming environment. Journal of Interactive Advertising, 5(1), 37–45.Google Scholar
  10. Cunningham, D., & Wallraven, C. (2011). Experimental design: From user studies to psychophysics. Natick: A K Peters.Google Scholar
  11. De Graef, P., Christiaens, D., & d’Ydewalle, G. (1990). Perceptual effects of scene context on object identification. Psychological Research, 52(4), 317–329.CrossRefGoogle Scholar
  12. Duchowski, A. T. (2003). Eye tracking methodology: Theory and practice. New York: Springer.MATHCrossRefGoogle Scholar
  13. Duncan, J. (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology. General, 113(4), 501–517.CrossRefGoogle Scholar
  14. Elazary, L., & Itti, L. (2008). Interesting objects are visually salient. Journal of Vision, 8(3:3), 1–15.Google Scholar
  15. El-Nasr, M. S., & Yan, S. (2006). Visual attention in 3D video games. In ACE 06: Proceedings of the 2006 ACM SIGCHI international conference on advances in computer entertainment technology (p. 22). New York: ACM.Google Scholar
  16. Eriksen, C., & St James, J. (1986). Visual attention within and around the field of focal attention: A zoom lens model. Attention, Perception, & Psychophysics, 40, 225–240.CrossRefGoogle Scholar
  17. Haber, J., Myszkowski, K., Yamauchi, H., & Seidel, H.-P. (2001). Perceptually guided corrective splatting. Computer Graphics Forum, 20(3), 142–152.CrossRefGoogle Scholar
  18. Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 478–500.CrossRefGoogle Scholar
  19. Hayhoe, M. M., Shrivastava, A., Mruczek, R., & Pelz, J. B. (2003). Visual memory and motor planning in a natural task. Journal of Vision, 3(1), 49–63.CrossRefGoogle Scholar
  20. Henderson, J. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences, 7(11), 498–504.CrossRefGoogle Scholar
  21. Henderson, J. M., Weeks, P. A., & Hollingworth, A. (1999). The effects of semantic consistency on eye movements during complex scene viewing. Journal of Experimental Psychology Human Perception & Performance, 25, 210–228.CrossRefGoogle Scholar
  22. Hillaire, S., Lécuyer, A., Cozot, R., & Casiez, G. (2008). Using an eye-tracking system to improve camera motions and depth-of-field Blur Effects in Virtual Environments. VR (pp. 47–50).Google Scholar
  23. Hillaire, S., Breton, G., Ouarti, N., Cozot, R., & Lécuyer, A. (2010). Using a visual attention model to improve gaze tracking systems in interactive 3D applications. Computer Graphics Forum, 29(6), 1830–1841.CrossRefGoogle Scholar
  24. Hornof, A., Cavender, A., & Hoselton, R. (2003). Eyedraw: A system for drawing pictures with eye movements. SIGACCESS Accessibility Computers (pp. 86–93), Atlanta, GA, USA.Google Scholar
  25. Isokoski, P., Joos, M., Spakov, O., & Martin, B. (2009). Gaze controlled games. Universal Access in the Information Society, 8, 323–337.CrossRefGoogle Scholar
  26. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transaction on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.CrossRefGoogle Scholar
  27. Itti, L., Dhavale, N., & Pighin, F. (2006). Photorealistic attention-based Gaze Animation. In Proceedings of the IEEE international conference on multimedia and expo (pp. 521–524). Toronto, Ontario, CanadaGoogle Scholar
  28. Jacob, R. J. K., & Karn, K. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In J. Hyönä, R. Radach, & H. Deubel (Eds.), The mind’s eye: Cognitive and applied aspects of eye movement research (pp. 573–605). Amsterdam: Elsevier.Google Scholar
  29. James, W., & Anonymous. (1890). The principles of psychology, Vol. 1, volume reprint edition. New York: Dover Publications.Google Scholar
  30. Jie, L., & Clark, J. J. (2007). Game design guided by visual attention. In L. Ma, M. Rauterberg, & R. Nakatsu (Eds.), Entertainment computing, ICEC 2007 in Lecture Notes in Computer Science (pp. 345–355), Shanghai: Springer.Google Scholar
  31. Kenny, A., Koesling, H., Delaney, D., McLoone, S., & Ward, T. (2005). A Preliminary investigation into eye gaze data in a first person shooter game. In 19th European Conference on Modelling and Simulation, Riga.Google Scholar
  32. Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4, 219–227.Google Scholar
  33. Komogortsev, O., & Khan, J. (2006). Perceptual attention focus prediction for multiple viewers in case of multimedia perceptual compression with feedback delay. In ETRA ’06: Proceedings of the 2006 symposium on eye tracking research & applications (pp. 101–108). New York: ACM.Google Scholar
  34. LaBerge, D. (1983). Spatial extent of attention to letters and words. Journal of Experimental Psychology: Human Perception and Performance, 9(3), 371–379.CrossRefGoogle Scholar
  35. Land, M., Mennie, N., & Rusted, J. (1999). The roles of vision and eye movements in the control of activities of daily living. Perception, 28(11), 1311–1328.CrossRefGoogle Scholar
  36. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., & Gross, M. (2010, July). Nonlinear disparity mapping for stereoscopic 3D. ACM Transaction on Graphics, 29(4), 1–75. doi:, URL: New York: ACM.Google Scholar
  37. Luebke, D., Hallen, B., Newfield, D., & Watson, B. (2000). Perceptually driven simplification using gaze-directed rendering.Google Scholar
  38. Marmitt, G., & Duchowski, A. T. (2002). Modeling visual attention in VR: Measuring the accuracy of predicted scanpaths. In Eurographics 2002, Short Presentations (pp. 217–226). Saarbrücken, Germany.Google Scholar
  39. McDonnell, R., Larkin, M., Hernández, B., Rudomin, I., & O’Sullivan, C. (2009). Eye-catching crowds: saliency based selective variation. ACM Transactions on Graphics, 28, 55:1–55:10.CrossRefGoogle Scholar
  40. Murphy, H., & Duchowski, A. T. (2001). Gaze-contingent level of detail rendering. In Proceedings of EuroGraphics 2001 (Short Papers). EuroGraphics Association. Manchester, England.Google Scholar
  41. Nacke, L., Lindley, C., & Stellmach, S. (2008). Log who’s playing: Psychophysiological game analysis made easy through event logging. In P. Markopoulos, B. de Ruyter, W. IJsselsteijn, & D. Rowland (Eds.), Fun and games in lecture notes in computer science (pp. 150–157). Berlin/Heidelberg: Springer.  10.1007/978-3-540-88322-715.
  42. Nacke, L., Stellmach, S., Sasse, D., & Lindley C. A. (2009). Gameplay experience in a gaze interaction game. In A. Villanueva, J. P. Hansen, & B. K. Ersbōll (Eds.) Proceedings of the 5th conference on communication by Gaze Interaction Ð COGAIN 2009: Gaze Interaction for Those Who Want It Most (pp. 49–54), Lyngby, Denmark. The COGAIN Association.Google Scholar
  43. Nacke, L. E., Stellmach, S., Sasse, D., Niesenhaus, J., & Dachselt, R. (2011). LAIF: A logging and interaction framework for gaze-based interfaces in virtual entertainment environments. Entertainment Computing, 2(4), 265–273. <ce:title>Special Section: International Conference on Entertainment Computing and Special Section: Entertainment Interfaces</ce:title>.Google Scholar
  44. Navalpakkam, V., & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45(2), 205–231.CrossRefGoogle Scholar
  45. O’Sullivan, C. (2005). Collisions and attention. ACM Transactions on Applied Perception, 2(3), 309–321.MathSciNetCrossRefGoogle Scholar
  46. Oliva, A., Torralba, A., Castelhano M. S., & Henderson, J. M. (2003). Top-down control of visual attention in object detection. In Proceedings of the IEEE International Conference on Image Processing (ICIP ’03). Barcelona, Catalonia, Spain.Google Scholar
  47. Palmer, S. E. (1999). Vision science: Photons to phenomenology. Boston: MIT Press.Google Scholar
  48. Pelz, J. B., & Canosa, R. (2001). Oculomotor behavior and perceptual strategies in complex tasks. Vision Research, 41, 3587–3596.CrossRefGoogle Scholar
  49. Peters, R. J., & Itti, L. (2008). Applying computational tools to predict gaze direction in interactive visual environments. ACM Transactions on Applied Perception, 5(2), 1–19.CrossRefGoogle Scholar
  50. Poole, A., & Ball, L. J. (2005). Eye tracking in human-computer interaction and usability research: Current status and future prospects. In C. Ghaoui (Ed.), Encyclopedia of human-computer interaction. Pennsylvania: Idea Group, Inc.Google Scholar
  51. Rahardja, S., Farbiz, F., Manders, C., Zhiyong, H., Ling, J. N. S., Khan, I. R., Ping, O. E., & Peng, S. (2009). Eye HDR: Gaze-adaptive system for displaying high-dynamic-range images. ACM SIGGRAPH ASIA 2009 Art Gallery & Emerging Technologies: Adaptation in SIGGRAPH ASIA ’09 (pp. 68–68). New York: ACM.Google Scholar
  52. Ramloll, R., Trepagnier, C., Sebrechts, M., & Beedasy, J. (2004). Gaze data visualization tools: opportunities and challenges. In Information Visualisation, 2004. IV 2004. Proceedings. Eighth International Conference on (pp. 173–180). London, UKGoogle Scholar
  53. Rothkopf, C. A., & Pelz, J. B. (2004). Head movement estimation for wearable eye tracker. In Proceedings of the 2004 symposium on eye tracking research & applications in ETRA ’04 (pp. 123–130). New York: ACM.Google Scholar
  54. Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. M. (2007). Task and context determine where you look. Journal of Vision, 7(14), 1–20.CrossRefGoogle Scholar
  55. Saito, T., & Takahashi, T. (1990). Comprehensible rendering of 3-D shapes. SIGGRAPH Computation Graphics, 24(4), 197–206.CrossRefGoogle Scholar
  56. Salvucci, D. D., & Goldberg, J. H. (2000). Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 symposium on eye tracking research & applications in ETRA ’00 (pp. 71–78). New York: ACM.Google Scholar
  57. Sasse D. (2008). A framework for psychophysiological data acquisition in digital games. Master’s thesis, Otto-von-Guericke-University Magdeburg, Magdeburg.Google Scholar
  58. Sennersten, C. (2004). Eye movements in an action game tutorial. Master’s thesis, Lund University, Lund.Google Scholar
  59. Sennersten, C., & Lindley, C. (2008). Evaluation of real-time eye gaze logging by a 3D game engine. In 12th IMEKO TC1 & TC7 joint symposium on man science and measurement (pp. 161–168). Annecy, France.Google Scholar
  60. Sennersten, C., & Lindley, C. (2009). An investigation of visual attention in FPS computer gameplay. In Conference in games and virtual worlds for serious applications, VS-GAMES ’09 (pp. 68–75). Coventry, UK.Google Scholar
  61. Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional blindness for dynamic events. Perception, 28, 1059–1074.CrossRefGoogle Scholar
  62. Snowden, R., Thompson, P., & Troscianko, T. (2006). Basic vision: An introduction to visual perception. Oxford University Press, USA.Google Scholar
  63. Starker, I., & Bolt, R. A. (1990). A gaze-responsive self-disclosing display. In CHI ’90: Proceedings of the SIGCHI conference on human factors in computing systems (pp. 3–10). New York: ACM.Google Scholar
  64. Stellmach, S. (2007). A psychophysiological logging system for a digital game modification. Unpublished Internship Report, Department of Simulation and Graphics. Otto-von-Guericke-University, Magdeburg.Google Scholar
  65. Stellmach S. (2009). Visual analysis of Gaze Data in virtual environments. Master’s thesis, Otto-von-Guericke-University Magdeburg, Magdeburg.Google Scholar
  66. Stellmach, S., Nacke, L., & Dachselt, R. (2010a). Advanced gaze visualizations for three-dimensional virtual environments. In Proceedings of the 2010 symposium on eye-tracking research & Applications in ETRA ’10 (pp. 109–112). New York: ACM.Google Scholar
  67. Stellmach, S., Nacke, L., & Dachselt, R. (2010b). 3D attentional maps: Aggregated gaze visualizations in three-dimensional virtual environments. In Proceedings of the international conference on advanced visual interfaces in AVI ’10 (pp. 345–348). New York: ACM.Google Scholar
  68. Stellmach, S., Nacke, L. E., Dachselt R., & Lindley C. A. (2010c). Trends and techniques in visual gaze analysis. CoRR, abs/1004.0258.Google Scholar
  69. Sundstedt, V. (2007). Rendering and validation of high-fidelity graphics using region-of-interest. PhD thesis, University of Bristol, Bristol.Google Scholar
  70. Sundstedt, V. (2010). Gazing at games: Using eye tracking to control virtual characters. ACM SIGGRAPH 2010 Courses in SIGGRAPH ’10 (pp. 5:1–5:160). New York: ACM.Google Scholar
  71. Sundstedt, V., Gutierrez, D., Anson, O., Banterle, F., & Chalmers, A. (2007). Perceptual rendering of participating media. ACM Transaction on Applied Perception, 4(3), 15.CrossRefGoogle Scholar
  72. Sundstedt, V., Stavrakis, E., Wimmer, M., & Reinhard, E. (2008). A psychophysical study of fixation behavior in a computer game. In APGV ’08: Proceedings of the 5th symposium on applied perception in graphics and visualization (pp. 43–50). New York: ACM.Google Scholar
  73. Sundstedt, V., Whitton, M., & Bloj, M. (2009). The whys, how tos, and pitfalls of user studies. ACM SIGGRAPH 2009 Courses in SIGGRAPH ’09 (pp. 25:1–25:205). New York: ACM.Google Scholar
  74. Tobii. (2006). User manual: Tobii eye tracker, ClearView analysis software.Google Scholar
  75. Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136.CrossRefGoogle Scholar
  76. van Zoest, W., & Donk, M. (2004). Bottom-up and top-down control in visual search. Perception, 33, 927–937.CrossRefGoogle Scholar
  77. Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Psychonomic Bulletin and Review, 1(2), 202–238.CrossRefGoogle Scholar
  78. Wolfe, J. (2000). Visual attention. In K. K. De Valois (Ed.), Seeing (pp. 335–386). San Diego: Academic.CrossRefGoogle Scholar
  79. Wolfe, J. M. (2007). Guided Search 4.0: Current Progress with a model of visual search. In Gray, W. (Ed.), Integrated models of cognitive systems (pp. 99–119). New York: Oxford University Press.CrossRefGoogle Scholar
  80. Wooding, D. S. (2002). Fixation maps: Quantifying eye-movement traces. Proceedings of the 2002 symposium on eye tracking research & applications in ETRA ’02 (pp. 31–36). New York: ACM.Google Scholar
  81. Yarbus, A. L. (1967). Eye movements during perception of complex objects. In Eye movements and vision (pp. 171–196). New York: Plenum Press.Google Scholar
  82. Yee, H., Pattanaik, S., & Greenberg, D. P. (2001). Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Transaction on Graphics, 20(1), 39–65.CrossRefGoogle Scholar
  83. Zammitto, V., Seif El-Nasr, M., & Newton, P. (2010). Exploring quantitative methods for evaluating sports games. In CHI 2010 workshop on brain, body and bytes: Psychophysiological user interaction.Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Veronica Sundstedt
    • 1
  • Matthias Bernhard
    • 2
  • Efstathios Stavrakis
    • 3
  • Erik Reinhard
    • 4
  • Michael Wimmer
    • 2
  1. 1.Blekinge Institute of TechnologySchool of ComputingKarlskronaSweden
  2. 2.Institute for Computer Graphics and AlgorithmsVienna University of TechnologyViennaAustria
  3. 3.Department of Computer ScienceUniversity of CyprusNicosiaCyprus
  4. 4.Department for Computer GraphicsMax Planck Institute for InformaticsSaarbrückenGermany

Personalised recommendations