The Attentive Hearing Aid: Eye Selection of Auditory Sources for Hearing Impaired Users

  • Jamie Hart
  • Dumitru Onceanu
  • Changuk Sohn
  • Doug Wightman
  • Roel Vertegaal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5726)


An often-heard complaint about hearing aids is that their amplification of environmental noise makes it difficult for users to focus on one particular speaker. In this paper, we present a new prototype Attentive Hearing Aid (AHA) based on ViewPointer, a wearable calibration-free eye tracker. With AHA, users need only look at the person they are listening to, to amplify that voice in their hearing aid. We present a preliminary evaluation of the use of eye input by hearing impaired users for switching between simultaneous speakers. We compared eye input with manual source selection through pointing and remote control buttons. Results show eye input was 73% faster than selection by pointing and 58% faster than button selection. In terms of recall of the material presented, eye input performed 80% better than traditional hearing aids, 54% better than buttons, and 37% better than pointing. Participants rated eye input as highest in the “easiest”, “most natural”, and “best overall” categories.


Eye Tracking Attentive User Interface Assistive Technology Hearing Impairment Input Devices Multi-Modal Input 


  1. 1.
    Arons, B.: Review of the Cocktail Party Effect. Journal of the American Voice I/O Society 12, 35–50 (1992)Google Scholar
  2. 2.
    Basu, S., Pentland, A.: Smart Headphones. In: Ext. Abstracts of CHI 2001, Seattle, pp. 267–268 (2001)Google Scholar
  3. 3.
    Boyer, D.J.: The Relationship Among Eye Movements, Head Movement, and Manual Responses in a Simulated Air Traffic Control Task. Technical Report, US Department of Transportation, FAA (1995)Google Scholar
  4. 4.
    Cycling 1974. Max/MSP/Jitter,
  5. 5.
    Cherry, E.C.: Some Experiments on the Recognition of Speech, with One and with Two Ears. Journal of Acoustic Society of America 25, 975–979 (1953)CrossRefGoogle Scholar
  6. 6.
    Dickie, C., Hart, J., Vertegaal, R., Eiser, A.: LookPoint: An Evaluation of Eye Input for Hands-Free Switching of Input Devices between Multiple Computers. In: Proc. of OzCHI 2006. ACM Press, New York (2006)Google Scholar
  7. 7.
    Duchowski, A.: Eye Tracking Methodology: Theory & Practice. Springer, London (2003)CrossRefzbMATHGoogle Scholar
  8. 8.
    Fono, D., Vertegaal, R.: EyeWindows: Evaluation of Eye-Controlled Zooming Windows for Focus Selection. In: Proc. of CHI 2005, pp. 151–160. ACM Press, New York (2005)Google Scholar
  9. 9.
    Greenberg, S., Fitchett, C.: Phidgets: Easy Development of Physical Interfaces through Physical Widgets. In: Proc. of UIST 2001. ACM Press, New York (2001)Google Scholar
  10. 10.
    Hawkins, D.B.: Comparisons of Speech Recognition in Noise by Mildly-to-Moderately Hearing-Impaired Children Using Hearing Aids and FM Systems. Journal of Speech and Hearing Disorders 49, 409–418 (1984)CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Hearing Loss Education Center,
  13. 13.
    Jacob, R.J.K.: The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At Is What You Get. ACM Transactions on Information Systems 9(3), 152–169 (1991)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Kochkin, S.: Hearing Loss Population Tops 31 Million People. Hearing Review 12(7) (2005)Google Scholar
  15. 15.
    Kochkin, S., Rogin, C.: Quantifying the Obvious: The Impact of Hearing Aids on Quality of Life. The Hearing Review 7(1), 8–34 (2000)Google Scholar
  16. 16.
    Langolf, G.D., Chaffin, D.B., Foulke, J.A.: An investigation of Fitts’ law using a wide range of movement amplitudes. Journal of Motor Behavior 8, 113–128 (1976)CrossRefGoogle Scholar
  17. 17.
    Maschke, C., Rupp, T., Hecht, K.: The Influence of Stressors on Biochemical Reactions - A Review of Present Scientific Findings with Noise. International Journal of Hygiene and Environmental Health 203, 45–53 (2000)CrossRefGoogle Scholar
  18. 18.
    Morrison, H.B., Casali, J.G.: Intelligibility of Synthesized Voice Messages in Commercial Truck Cab Noise for Normal-Hearing and Hearing-Impaired Listeners. International Journal of Speech Technology 2, 33–44 (1997)CrossRefGoogle Scholar
  19. 19.
    Nickel, K., Stiefelhagen, R.: Pointing Gesture Recognition on 3d-Tracking of Face, Hands and Head Orientation. In: Proc. ICMI, pp. 380–400 (2003)Google Scholar
  20. 20.
    O’Neill, G., Summer, L., Shirey, L.: Hearing Loss: A Growing Problem that Affects Quality of Life. Challenges for the 21st Century: Chronic and Disabling Conditions 2, 1–6 (1999)Google Scholar
  21. 21.
    Ricketts, T.A., Mueller, H.G.: Making Sense of Directional Microphone Hearing Aids. American Journal of Audiology 8(2), 117–128 (1999)CrossRefGoogle Scholar
  22. 22.
    Ricketts, T.A., Dhar, S.: Comparison of Performance across Three Directional Hearing Aids. American Journal of Audiology 8(10), 180–189 (1999)Google Scholar
  23. 23.
    Selker, T., Lockerd, A., Martinez, J.: Eye-R, A Glasses-Mounted Eye Motion Detection Interface. In: Ext. Abstracts of CHI 2001, pp. 179–180. ACM Press, New York (2001)Google Scholar
  24. 24.
    Sibert, L.E., Jacob, R.J.K.: Evaluation of Eye Gaze Interaction. In: Proc. of CHI 2000. ACM, New York (2000)Google Scholar
  25. 25.
    Smith, A.P.: Noise, Performance Efficiency and Safety. International Archives of Occupational and Environmental Health 62, 1–5 (1990)CrossRefGoogle Scholar
  26. 26.
    Smith, A.P.: Noise and Aspects of Attention. British Journal of Psychology 82, 313–324 (1991)CrossRefGoogle Scholar
  27. 27.
    Smith, J.D.: ViewPointer: Lightweight Calibration-Free Eye Tracking for Ubiquitous Handsfree Deixis. Master’s Thesis, Queen’s University (2005)Google Scholar
  28. 28.
    Smith, J.D., Vertegaal, R., Sohn, C.: ViewPointer: Lightweight Calibration-Free Eye Tracking for Ubiquitous Handsfree Deixis. In: Proc. of UIST 2005, pp. 53–61. ACM, New York (2005)Google Scholar
  29. 29.
    Tognazzini, B.: Tog on Interface. Addison-Wesley Longman Publishing Co., Boston (1992)Google Scholar
  30. 30.
    Valente, M., Fabry, D.A., Potts, L.G.: Recognition of Speech in Noise with Hearing Aids using Dual Microphones. Journal of the American Academy of Audiology 6(6), 440–449 (1995)Google Scholar
  31. 31.
    Varibel Hearing Glasses,
  32. 32.
    Velichkovsky, B.M., Hansen, J.P.: New Technological Windows into Mind: There is More in Eyes and Brain for Human-Computer Interaction. In: Proc. of CHI 1996, pp. 496–503. ACM Press, New York (1996)Google Scholar
  33. 33.
    Vertegaal, R., Ding, Y.: Explaining Effects of Eye Gaze on Mediated Group Conversations: Amount or synchronization? In: Proceedings of CSCW 2002, pp. 41–48. ACM Press, New Orleans (2002)Google Scholar
  34. 34.
    Vertegaal, R., Slagter, R., van der Veer, G., Nijholt, A.: Eye Gaze Patterns in Conversations: There is More to Conversational Agents Than Meets the Eyes. In: Proc. of CHI 2001, pp. 301–308. ACM, New York (2001)Google Scholar
  35. 35.
    Ware, C., Mikaelian, H.H.: An Evaluation of an Eye Tracker as a Device for Computer Input. In: Proc. of CHI+GI 1987, pp. 183–188. ACM Press, New York (1987)Google Scholar
  36. 36.
    Wang, J., Zhai, S., Su, H.: Chinese Input with Keyboard and Eye-Tracking - An Anatomical Study. In: Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems, pp. 349–356 (2001)Google Scholar
  37. 37.
    Watanabe, J., Nii, H., Hashimoto, Y., Inami, M.: Visual Resonator: Interface for Interactive Cocktail Party Phenomenon. In: Ext. Abstracts of CHI 2006, pp. 1505–1510. ACM Press, New York (2006)Google Scholar
  38. 38.
    Zhai, S., Morimoto, C., Ihde, S.: Manual and gaze input cascaded (MAGIC) pointing. In: Proceedings of the ACM CHI 1999, pp. 246–253 (1999)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2009

Authors and Affiliations

  • Jamie Hart
    • 1
  • Dumitru Onceanu
    • 1
  • Changuk Sohn
    • 1
  • Doug Wightman
    • 1
  • Roel Vertegaal
    • 1
  1. 1.Human Media LabQueen’s UniversityKingstonCanada

Personalised recommendations