A Study on Gestural Interaction with a 3D Audio Display

  • Georgios Marentakis
  • Stephen A. Brewster
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3160)


The study reported here investigates the design and evaluation of a gesture-controlled, spatially-arranged auditory user interface for a mobile computer. Such an interface may provide a solution to the problem of limited screen space in handheld devices and lead to an effective interface for mobile/eyes-free computing. To better understand how we might design such an interface, our study compared three potential interaction techniques: head nodding, pointing with a finger and pointing on a touch tablet to select an item in exocentric 3D audio space. The effects of sound direction and interaction technique on the browsing and selection process were analyzed. An estimate of the size of the minimum selection area that would allow efficient 3D sound selection is provided for each interaction technique. Browsing using the touch screen was found to be more accurate than the other two techniques, but participants found it significantly harder to use.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arons, B.: A Review of the Cocktail Party Effect. Journal of the American Voice I/O Society 12, 35–50 (1992)Google Scholar
  2. 2.
    Blattner, M.M., Sumikawa, D.A., Greenberg, R.M.: Earcons and Icons: Their Structure and Common Design Principles. Human-Computer Interaction 4(1), 11–44 (1989)CrossRefGoogle Scholar
  3. 3.
    Blauert, J.: Spatial Hearing: The psychophysics of human sound localization. MIT Press, Cambridge (1999)Google Scholar
  4. 4.
    Brewster, S., Lumsden, J., Bell, M., Hall, M., Tasker, S.: Multimodal ’Eyes-Free’ Interaction Techniques for Wearable Devices. In: ACM CHI, pp. 463–480. ACM Press, Addison-Wesley, Fort Lauderdale, FL (2003)Google Scholar
  5. 5.
    Brewster, S.A.: The design of sonically-enhanced widgets. Interacting with Computers 11(2), 211–235 (1998)CrossRefGoogle Scholar
  6. 6.
    Cohen, M.: Throwing, pitching and catching sound: audio windowing models and modes. International Journal of Man - Machine Studies 39, 269–304 (1993)CrossRefGoogle Scholar
  7. 7.
    Cohen, M., Ludwig, L.: Multidimensional Audio Window Management. Internaltional Journal of Man - Machine Studies 34, 319–336 (1991)CrossRefGoogle Scholar
  8. 8.
    Gaver, W.W.: The SonicFinder: An Interface that uses Auditory Icons. Human-Computer Interaction 4, 67–94 (1989)CrossRefGoogle Scholar
  9. 9.
    Goose, S., Moller, C.: A 3D Audio Only Interactive Web Browser: Using Spatialization to Convey Hypermedia Document Structure. In: 7th ACM international conference on Multimedia, pp. 363–371. ACM Press, Orlando (1999)CrossRefGoogle Scholar
  10. 10.
    Leek, M.R.: Adaptive procedures in psychophysical research. Journal of Perception & Psychophysics 63(8), 1279–1292 (2001)CrossRefGoogle Scholar
  11. 11.
    McGookin, D.K., Brewster, S.A.: An Investigation into the Identification of Concurrently Presented Earcons. In: ICAD 2003, Boston, MA, pp. 42–46 (2003)Google Scholar
  12. 12.
    Pirhonen, A., Brewster, S., Holguin, C.: Gestural and audio metaphors as a means of control for mobile devices. In: ACM CHI, Minneapolis, Minnesota, USA, pp. 291–298. ACM Press, New York (2002)Google Scholar
  13. 13.
    Savidis, A., Stephanidis, C., Korte, A., Rispien, K., Fellbaum, C.: A generic directmanipulation 3D auditory environment for hierarchical navigation in non-visual interaction. In: ACM ASSETS 1996, pp. 117–123. ACM Press, Vancouver (1996)Google Scholar
  14. 14.
    Sawhney, N., Schmandt, C.: Nomadic Radio: Speech and Audio Interaction for Contextual Messaging in Nomadic Environments. ACM Transactions on Computer-Human Interaction 7(3), 353–383 (2000)CrossRefGoogle Scholar
  15. 15.
    Gaver, W.W., Smith, R.B.: Effective Sounds in Complex Systems: The ARKOLA Simulation. In: Conference on Human Factors in Computing Systems archive, ACMCHI, New Orleans, Louisiana, United States, pp. 85–90. ACM Press, New York (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Georgios Marentakis
    • 1
  • Stephen A. Brewster
    • 1
  1. 1.Glasgow Interactive Systems Group Department of Computing ScienceUniversity of GlasgowGlasgowUK

Personalised recommendations