Beyond Direct Gaze Typing: A Predictive Graphic User Interface for Writing and Communicating by Gaze

  • Maria Laura MeleEmail author
  • Damon Millar
  • Christiaan Erik Rijnders
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9170)


This paper introduces a new gaze-based Graphic User Interface (GUI) for Augmentative and Alternative Communication (AAC). In the state of the art, prediction methods to accelerate the production of textual, iconic and pictorial communication only by gaze control are still needed. The proposed GUI translates gaze inputs into words, phrases or symbols by the following methods and techniques: (i) a gaze-based information visualization technique, (ii) a prediction technique combining concurrent and retrospective methods, and (iii) an alternative prediction method based either on the recognition or morphing of spatial features. The system is designed for extending the communication function of individuals with severe motor disabilities, with the aim to allow end-users to independently hold a conversation without needing a human interpreter.


Augmentative and alternative communication Graphical user interfaces for disabled users Gaze based interaction Assistive technologies 


  1. 1.
    Millar, S., Scott, J.: What is augmentative and alternative communication? An introduction. Augmentative communication in practice 2. University of Edinburgh CALL Centre, Edinburgh (1998)Google Scholar
  2. 2.
    Lancioni, G.E., Singh, N.N., O’Reilly, M.F., Sigafoos, J., Oliva, D., Basili, G.: New rehabilitation opportunities for persons with multiple disabilities through the use of microswitch technology. In: Federici, S., Scherer, M.J. (eds.) Assistive Technology Assessment Handbook, pp. 399–419. CRC Press, New York (2012)Google Scholar
  3. 3.
    Borsci, S., Kurosu, M., Federici, S., Mele, M.L.: Computer Systems Experiences Of Users With And Without Disabilities: An Evaluation Guide For Professionals. CRC Press, Boca Raton (2013)Google Scholar
  4. 4.
    Borsci, S., Kurosu, M., Federici, S., Mele, M.L.: Systemic User Experience. In: Federici, S., Scherer, M.J. (eds.) Assistive Technology Assessment Handbook, pp. 337–359. CRC Press, Boca Raton (2012)Google Scholar
  5. 5.
    Federici, S., Scherer, M.J.: The Assistive Technology Assessment Model and Basic Definitions. In: Federici, S., Scherer, M.J. (eds.) Assistive Technology Assessment Handbook, pp. 1–10. CRC Press, Boca Raton (2012)Google Scholar
  6. 6.
    Federici, S., Corradi, F., Mele, M.L., Miesenberger, K.: From cognitive ergonomist to psychotechnologist: a new professional profile in a multidisciplinary team in a centre for technical aids. In: Gelderblom, G.J., Soede, M., Adriaens, L., Miesenberger, K. (eds.) Everyday Technology for Independence and Care: AAATE 2011, vol. 29, pp. 1178–1184. IOS Press, Amsterdam (2011)Google Scholar
  7. 7.
    Mele, M.L., Federici, S.: Gaze and eye-tracking solutions for psychological research. Cogn. Process. 13(1), 261–265 (2012)CrossRefGoogle Scholar
  8. 8.
    Mele, M.L., Federici, S.: A psychotechnological review on eye-tracking systems: towards user experience. Disabil. Rehabil. Assist. Technol. 7(4), 261–281 (2012)CrossRefGoogle Scholar
  9. 9.
    Federici, S., Corradi, F., Meloni, F., Borsci, S., Mele, M.L., Dandini de Sylva, S., Scherer, M.J.: A person-centered assistive technology service delivery model: a framework for device selection and assignment. Life Span Disabil. 17(2), 175–198 (2014)Google Scholar
  10. 10.
    Federci, S., Borsci, S., Mele, M.L.: Environmental evaluation of a rehabilitation aid interaction under the framework of the ideal model of assistive technology assessment process. In: Kurosu, M. (ed.) HCII/HCI 2013, Part I. LNCS, vol. 8004, pp. 203–210. Springer, Heidelberg (2013)Google Scholar
  11. 11.
    Federici, S., Borsci, S.: Providing assistive technology in Italy: the perceived delivery process quality as affecting abandonment. Disabil. Rehabil. Assist. Technol. 1–10 (2014) Google Scholar
  12. 12.
    Bates, R., Donegan, M., Istance, H.O., Hansen, J.P., Räihä, K.J.: Introducing COGAIN: communication by gaze interaction. Universal Access Inf. 6(2), 159–166 (2007)CrossRefGoogle Scholar
  13. 13.
    Leigh, R.J., Zee, D.S.: The Neurology of Eye Movements, vol. 90. Oxford University Press, New York (1999)Google Scholar
  14. 14.
    Saito, S.: Does fatigue exist in a quantitative measurement of eye movements? Ergonomics 35(5/6), 607–615 (1992)CrossRefGoogle Scholar
  15. 15.
    Majaranta, P.: Communication and text entry by gaze. In: Majaranta, P., Aoki, H., Donegan, M., Hansen, D.W., Hansen, J.P., Hyrskykari, A., Räihä, K.J. (eds.) Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies, pp. 63–77. IGI Global, Hershey (2012)CrossRefGoogle Scholar
  16. 16.
    Yarbus, A.L.: Eye Movements and Vision. Plenum Press, New York (1967)CrossRefGoogle Scholar
  17. 17.
    Itti, L., Koch, C., Niebur, E.: A model of saliency based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  18. 18.
    Kintsch, W.: Models for Free Recall and Recognition. Models of Human Memory, vol. 124. Academic Press, New York (1970)Google Scholar
  19. 19.
    Bouma, H.: Visual interference in the parafoveal recognition of initial and final letters of words. Vision. Res. 13, 762–782 (1973)CrossRefGoogle Scholar
  20. 20.
    Bouwhuis, D., Bouma, H.: Visual word recognition of three letter words as derived from the recognition of the constituent letters. Percept. Psychophys. 25, 12–22 (1979)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maria Laura Mele
    • 1
    • 2
    Email author
  • Damon Millar
    • 2
  • Christiaan Erik Rijnders
    • 2
  1. 1.ECONAInteruniversity Centre for Research on Cognitive Processing in Natural and Artificial Systems, Sapienza University of RomeRomeItaly
  2. 2.COGISEN Engineering CompanyRomeItaly

Personalised recommendations