Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

What Are You Looking at?

Improving Visual Gaze Estimation by Saliency


In this paper we present a novel mechanism to obtain enhanced gaze estimation for subjects looking at a scene or an image. The system makes use of prior knowledge about the scene (e.g. an image on a computer screen), to define a probability map of the scene the subject is gazing at, in order to find the most probable location. The proposed system helps in correcting the fixations which are erroneously estimated by the gaze estimation device by employing a saliency framework to adjust the resulting gaze point vector. The system is tested on three scenarios: using eye tracking data, enhancing a low accuracy webcam based eye tracker, and using a head pose tracker. The correlation between the subjects in the commercial eye tracking data is improved by an average of 13.91%. The correlation on the low accuracy eye gaze tracker is improved by 59.85%, and for the head pose tracker we obtain an improvement of 10.23%. These results show the potential of the system as a way to enhance and self-calibrate different visual gaze estimation systems.


  1. Bates, R., Istance, H., Oosthuizen, L., & Majaranta, P. (2005). Survey of de-facto standards in eye tracking. In COGAIN conf. on comm. by gaze inter.

  2. Cristinacce, D., Cootes, T., & Scott, I. (2004). A multi-stage approach to facial feature detection. In BMVC.

  3. Einhauser, W., Spain, M., & Perona, P. (2008). Objects predict fixations better than early saliency. Journal of Vision, 8(14).

  4. Geisler, W. S., & Banks, M. S. (1995). Fundamentals, techniques and design: Vol. 1. Handbook of optics (2nd ed.). New York: McGraw-Hill.

  5. Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. PAMI, 32(3).

  6. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. PAMI, 20(11).

  7. Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009). Learning to predict where humans look. In ICCV.

  8. Kroon, B., Boughorbel, S., & Hanjalic, A. (2008). Accurate eye localization in webcam content. In FG.

  9. Langton, S. R., Honeyman, H., & Tessler, E. (2004). The influence of head contour and nose angle on the perception of eye-gaze direction. Perception & Psychophysics, 66(5).

  10. Liu, T., Sun, J., Zheng, N. N., Tang, X., & Shum, H. Y. (2007). Learning to detect a salient object. In CVPR.

  11. Ma, Y. F., & Zhang, H. J. (2003). Contrast-based image attention analysis by using fuzzy growing. In ACM MM.

  12. Murphy-Chutorian, E., & Trivedi, M. (2009). Head pose estimation in computer vision: A survey. PAMI, 31(4).

  13. Peters, R. J., Iyer, A., Koch, C., & Itti, L. (2005). Components of bottom-up gaze allocation in natural scenes. Journal of Vision, 5(8).

  14. Rossi, E. A., & Roorda, A. (2009). The relationship between visual resolution and cone spacing in the human fovea. Nature Neuroscience, 13.

  15. Smith, K., Ba, S. O., Odobez, J. M., & Gatica-Perez, D. (2008). Tracking the visual focus of attention for a varying number of wandering people. PAMI, 30(7).

  16. Spain, M., & Perona, P. (2008). Some objects are more equal than others: Measuring and predicting importance. In ECCV.

  17. Valenti, R., & Gevers, T. (2008). Accurate eye center location and tracking using isophote curvature. In CVPR.

  18. Valenti, R., Sebe, N., & Gevers, T. (2009). Image saliency by isocentric curvedness and color. In ICCV.

  19. Xiao, J., Kanade, T., & Cohn, J. (2002). Robust full motion recovery of head by dynamic templates and re-registration techniques. In FG.

  20. Zhu, J., & Yang, J. (2002). Subpixel eye gaze tracking. In FGR. Los Alamitos: IEEE Computer Society.

Download references

Author information

Correspondence to Roberto Valenti.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Valenti, R., Sebe, N. & Gevers, T. What Are You Looking at?. Int J Comput Vis 98, 324–334 (2012).

Download citation


  • HCI
  • Gaze estimation
  • Saliency
  • Head pose
  • Eye location