Perceptual Image Retrieval Using Eye Movements

  • Oyewole Oyekoya
  • Fred Stentiford
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4153)


This paper explores the feasibility of using an eye tracker as an image retrieval interface. A database of image similarity values between 1000 Corel images is used in the study. Results from participants performing image search tasks show that eye tracking data can be used to reach target images in fewer steps than by random selection. The effects of the intrinsic difficulty of finding images and the time allowed for successive selections were also investigated.


Image Retrieval Target Image Image Type Relevance Feedback Fixation Threshold 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Corno, F., Farinetti, L., Signorile, I.: A cost effective solution for eye-gaze assistive technology. In: IEEE Int. Conf. on Multimedia and Expo., Lausanne, August 26-29 (2002)Google Scholar
  2. 2.
    Cox, I.J., Miller, M.L., Minka, T.P., Papathomas, T.V., Yianilos, P.N.: The Bayesian image retrieval system, PicHunter: theory, implementation, and Psychophysical experiments. IEEE Trans. on Image Processing 9(1) (2000)Google Scholar
  3. 3.
    Duchowski, A.T.: A Breadth-First Survey of Eye Tracking Applications. Behaviour Research Methods, Instruments, & Computers (BRMIC) 34(4), 455–470 (2002)CrossRefGoogle Scholar
  4. 4.
    Fitts, P.M., Jones, R.E., Milton, J.L.: Eye Movement of Aircraft Pilots during Instrument-Landing Approaches. Aeronautical Engineering Review 9, 24–29 (1950)Google Scholar
  5. 5.
    Hansen, J.P., Anderson, A.W., Roed, P.: Eye gaze control of multimedia systems. In: Anzai, Y., Ogawa, K., Mori, H. (eds.) Symbiosis of Human and Artifact, vol. 20A, pp. 37–42. Elsevier Science, Amsterdam (1995)Google Scholar
  6. 6.
    Henderson, J.M., Hollingworth, A.: High-Level Scene Perception. Annual Reviews Psychology 50, 243–271 (1999)CrossRefGoogle Scholar
  7. 7.
    Itti, L.: Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans. on Image Processing 13(10), 1304–1318 (2004)CrossRefGoogle Scholar
  8. 8.
    LC Technologies Inc.,
  9. 9.
    Mackworth, N., Morandi, A.: The gaze selects informative details within pictures. Perception and Psychophysics 2, 547–552 (1967)CrossRefGoogle Scholar
  10. 10.
    McCarthy, J., Sasse, M.A., Riegelsberger, J.: Could I have the menu please? An eye tracking study of design conventions. In: Proceedings of HCI 2003, Bath, UK, September 8-12 (2003)Google Scholar
  11. 11.
    Multimedia Understanding through Semantics, Computation and Learning, Network of Excellence. EC 6th Framework Programme. FP6-507752,
  12. 12.
    Numajiri, T., Nakamura, A., Kuno, Y.: Speed browser controlled by eye movements. In: IEEE Int. Conf. on Multimedia and Expo., Lausanne, August 26-29 (2002)Google Scholar
  13. 13.
    Oyekoya, O.K., Stentiford, F.W.M.: Exploring Human Eye Behaviour Using a Model of Visual Attention. In: International Conference on Pattern Recognition, Cambridge, UK (August 2004)Google Scholar
  14. 14.
    Oyekoya, O.K., Stentiford, F.W.M.: A Performance Comparison of Eye Tracking and Mouse Interfaces in a Target Image Identification Task. In: 2nd European Workshop on the Integration of Knowledge, November 30 - December 1 (2005)Google Scholar
  15. 15.
    Oyekoya, O., Stentiford, F.: An eye tracking interface for image search. In: Eye Tracking Research & Applications symposium (ETRA 2006), San Diego (2006)Google Scholar
  16. 16.
    Puolamäki, K., Salojärvi, J., Savia, E., Simola, J., Kaski, S.: Combining Eye Movements and Collaborative Filtering for Proactive Information Retrieval. In: Proceedings of the 28th ACM Conference on Research and Development in Information Retrieval (SIGIR) (2005)Google Scholar
  17. 17.
    Stentiford, F. W. M.: Attention Based Similarity. Pattern Recognition (to appear, 2006) Google Scholar
  18. 18.
    Urban, J., Jose, J.M., van Rijsbergen, C.J.: An adaptive approach towards content-based image retrieval. In: Proc. of the Third International Workshop on Content-Based Multimedia In-dexing (CBMI 2003), pp. 119–126 (2003)Google Scholar
  19. 19.
    Venters, C.C., Eakins, J.P., Hartley, R.J.: The user interface and content based image re-trieval systems. In: Proc. of the 19th BCS-IRSG Research Colloquium, Aberdeen (April 1997)Google Scholar
  20. 20.
    Ward, D.J., MacKay, D.J.C.: Fast hands-free writing by gaze direction. Nature 418, 838 (2002)CrossRefGoogle Scholar
  21. 21.
    Yarbus, A.: Eye Movements and Vision. Plenum Press, New York (1967)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Oyewole Oyekoya
    • 1
  • Fred Stentiford
    • 1
  1. 1.University College London, Adastral ParkIpswichUnited Kingdom

Personalised recommendations