Gaze as a Supplementary Modality for Interacting with Ambient Intelligence Environments

  • Daniel Gepner
  • Jérôme Simonin
  • Noëlle Carbonell
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4555)


We present our current research on the implementation of gaze as an efficient and usable pointing modality supplementary to speech, for interacting with augmented objects in our daily environment or large displays, especially immersive virtual reality environments, such as reality centres and caves. We are also addressing issues relating to the use of gaze as the main interaction input modality. We have designed and developed two operational user interfaces: one for providing motor-disabled users with easy gaze-based access to map applications and graphical software; the other for iteratively testing and improving the usability of gaze-contingent displays.


gaze tracking gaze pointing gaze-contingent displays gaze and speech human-computer interaction multimodal human-computer interaction ambient intelligence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ashmore, M., Duchowski, A.T., Shoemaker, G.: Efficient Eye Pointing with a Fisheye Lens. In: Proceedings of the 2005 CHCCS/ACM Conference on Graphics Interface (GI 2005), University of Waterloo, Victoria, BC, May 9-11, 2005, pp. 203–210. School of Computer Science, Waterloo, Ontario (2005)Google Scholar
  2. 2.
    Bates, R., Istance, H.O.: Towards eye based virtual environment interaction for users with high-level motor disabilities. In: Proceedings of 5th International Conference Series on Disability, Virtual Reality and Associated Technologies (ICDVRAT 2004), Oxford, UK, September 20-22, 2004, pp. 276–282 (2004)Google Scholar
  3. 3.
    Beymer, D., Flickner, M.: Eye Gaze Tracking Using an Active Stereo Head. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2003), Madison, Wisconsin, June 16-22, 2003, vol. 2, pp. 451–458 (2003)Google Scholar
  4. 4.
    Bolt, R.A.: Put-that-there: Voice and gesture at the graphics interface. In: Proceedings of the 7th International Conference on Computer Graphics and Interactive Techniques, Seattle, WA, July 14-18, 1980, pp. 262–270. ACM Press, New York (1980)Google Scholar
  5. 5.
    Carbonell: Ambient Multimodality: towards Advancing Computer Accessibility and Assisted Living. International Journal on Universal Access in the Information Society 5(1), 96–104 (2006)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Cournia, N., Smith, J.D., Duchowski, A.T.: Gaze vs. Hand-Based Pointing in Virtual Environments. In: Proceedings of International ACM Conference on Human Factors in Computing Systems (CHI 2003), Ft. Lauderdale (FL), April 5-10, 2003, pp. 772–773. ACM Press, New York (2003) Short Talks and Interactive PostersCrossRefGoogle Scholar
  7. 7.
    Duchowski, A.T., Cournia, N., Murphy, H.: Gaze-Contingent Displays: a Review. Cyber Psychology and Behavior 7(6), 621–634 (2004)CrossRefGoogle Scholar
  8. 8.
    Hatfield, F., Jenkins, E.A., Calhoun, G.: Principles and guidelines for the design of eye/voice interaction dialogs. In: Proceedings of 3rd Symposium on Human Interaction with Complex Systems (HICS 1996), Dayton, Ohio, August 25-28, 1996, pp. 10–19 (1996)Google Scholar
  9. 9.
    Jacob, R.J.K.: Eye tracking in advanced interface design. In: Barfield, W., Furness, T.A. (eds.) Virtual Environments and Advanced Interface Design, pp. 258–288. Oxford University Press, Oxford (1995)Google Scholar
  10. 10.
    Park, K.R.: Vision-based Facial and Eye Gaze Tracking. In: Biundo, S., Frühwirth, T., Palm, G. (eds.) KI 2004. LNCS (LNAI), vol. 3238, pp. 453–465. Springer, Heidelberg (2004)Google Scholar
  11. 11.
    Marajanta, P., Räihä, K.-J.: Twenty years of eye typing: systems and design issues. In: Proceedings of the 2002 Symposium on Eye Tracking Research and Applications, New Orleans, Louisiana, March 25-27, 2002, pp. 15–22 (2002)Google Scholar
  12. 12.
    Oviatt, S.L: Mutual disambiguation of recognition errors in a multimodal architecture. In: Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 1999), Pittsburgh, May 15-20, 1999, pp. 576–583. ACM Press, New York (1999)CrossRefGoogle Scholar
  13. 13.
    Oyekoya, O.K., Stentiford, F.W.M.: Eye Tracking as a New Interface for Image Retrieval. Behavior and Technology Journal 22(3), 161–169 (2004)Google Scholar
  14. 14.
    Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in Eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Ressearch and Applications (ETRA 2000), Palm Beach Gardens, Florida, November 6-8, 2000, pp. 71–78. ACM Press, New York (2000)CrossRefGoogle Scholar
  15. 15.
    Simonin, J., Kieffer, S., Carbonell, N.: Effects of display layout on gaze activity during visual search. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1054–1058. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Smith, J.D., Graham, N.T.C.: Use of Eye Movements for Video Game Control (The ACM Digital Library). In: Proceedings of International ACM Conference on Advances in Computer Entertainment Technology (ACE 2006), Hollywood, CA, June 14-16, 2006, p. 8. ACM Press, New York (2006)Google Scholar
  17. 17.
    Tanriverdi, V., Jacob, R.J.K: Interacting with Eye Movements in Virtual Environments. In: Proceedings of International ACM Conference on Human Factors in Computing Systems (CHI 2000), The Hague, the Netherlands, April 1-6, 2003, pp. 265–272. ACM Press, New York (2000)CrossRefGoogle Scholar
  18. 18.
    Want, R., Farkas, K.I., Narayanaswami, C.: Guest Editors’ Introduction: Energy Harvesting and Conservation. IEEE Pervasive Computing 4(1), 14–17 (2005)CrossRefGoogle Scholar
  19. 19.
  20. 20.
    Zhang, Q., Imamiya, A.G.K., Mao, X.: A Gaze and Speech Multimodal Interface. In: Proceedings 24th International Conference on Distributed Computing Systems Workshops, Tokyo, Japan, March 23-26, 2004, pp. 208–213 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Daniel Gepner
    • 1
  • Jérôme Simonin
    • 1
  • Noëlle Carbonell
    • 1
  1. 1.LORIA, CNRS, INRIA & Nancy Université, Campus Scientifique, BP 239, F54506, Vandoeuvre-lès-Nancy CedexFrance

Personalised recommendations