The Ecological AUI (Auditory User Interface) Design and Evaluation of User Acceptance for Various Tasks on Smartphones

  • Myounghoon Jeon
  • Ju-Hwan Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8007)


With the rapid development of the touch screen technology, some usability issues of smartphones have been reported [1]. To tackle those user experience issues, there has been research on the use of non-speech sounds on the mobile devices [e.g., 2, 3-7]. However, most of them have focused on a single specific task of the device. Given the varying functions of the smartphone, the present study designed plausibly integrated auditory cues for diverse functions and evaluated user acceptance levels from the ecological interface design perspective. Results showed that sophisticated auditory design could change users’ preference and acceptance of the interface and the extent depended on usage contexts. Overall, participants gave significantly higher scores on the functional satisfaction and the fun scales in the sonically-enhanced smartphones than in the no-sound condition. The balanced sound design may free users from auditory pollution and allow them to use their devices more pleasantly.


Auditory user interface ecological user interface design smartphones user acceptance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Norman, D.A., Nielsen, J.: Gestural interfaces: A step backward in usability. Interactions 17(5), 46–49 (2010)CrossRefGoogle Scholar
  2. 2.
    Brewster, S.A., Leplâtre, G., Crease, M.G.: Using non-speech sounds in mobile computing devices. In: Proceedings of the 1st Mobile HCI 1998, Glasgow, UK (1998)Google Scholar
  3. 3.
    Leplâtre, G., Brewster, S.A.: Designing non-speech sounds to support navigation in mobile phone menus. In: Proceedings of the 6th ICAD 2000, Atlanta, GA, USA (2000)Google Scholar
  4. 4.
    Palladino, D., Walker, B.N.: Efficiency of spearcon-enhanced navigation of one dimensional electronic menus. In: Proceedings of the ICAD 2008, Paris, France (2008)Google Scholar
  5. 5.
    Brewster, S.A., Cryer, P.G.: Maximising screen-space on mobile computing devices. In: Proceedings of the ACM CHI 1999, Pittsburgh, PA, USA (1999)Google Scholar
  6. 6.
    Jeon, M., Walker, N.B.: “Spindex” (Speech Index) improves acceptance and performance in auditory menu navigation for visually impaired and sighted users. ACM Transactions on Accessible Computing 3(3), 10:1–26:1 (2011)CrossRefGoogle Scholar
  7. 7.
    Jeon, M., Walker, B.N., Srivastava, A.: “Spindex” (speech index) enhances menu navigation on touch screen devices with tapping, wheeling, and flicking gestures. ACM Transactions on Computer-Human Interaction 19(2), 14:1–27:1 (2012)Google Scholar
  8. 8.
    Oh, J.W., Park, J.H., Jo, J.H., Lee, C., Yun, M.H.: Development of a kansei analysis system on the physical user interface. In: Proceedings of the HCI, Kangwon, Korea (2007)Google Scholar
  9. 9.
    Rasmussen, J., Vicente, K.J.: Coping with human errors through system design: Implications for ecological interface design. International Journal of Man-Machine Studies 31, 517–534 (1989)CrossRefGoogle Scholar
  10. 10.
    Gaver, W.W.: Auditory icons: Using sound in computer interfaces. Human-Computer Interaction 2, 167–177 (1986)CrossRefGoogle Scholar
  11. 11.
    Wilson, J., Walker, B.N., Lindsay, J., Cambias, C., Dellaert, F.: SWAN: System for wearable audio navigation. In: Proceedings of the 11th ISWC 2007 (2007)Google Scholar
  12. 12.
    Blattner, M.M., Sumikawa, D.A., Greenberg, R.M.: Earcons and icons: Their structure and common design principles. Human-Computer Interaction 4, 11–44 (1989)CrossRefGoogle Scholar
  13. 13.
    Helle, S., Leplatre, G., Marila, J., Laine, P.: Menu sonification in a mobile phone – a prototype study. In: Proceedings of the ICAD 2001, Espoo, Finland (2001)Google Scholar
  14. 14.
    Brewster, S.A.: The design of sonically-enhanced widgets. Interacting with Computers 11(2), 211–235 (1998)CrossRefGoogle Scholar
  15. 15.
    Yalla, P., Walker, B.N.: Advanced auditory menus: Design and evaluation of auditory scroll bars. In: Proceedings of the ASSETS 2008 (2008)Google Scholar
  16. 16.
    McGee-Lennon, M., Wolters, M.K., McLachlan, R., Brewster, S., Hall, C.: Name that tune: Musicons as reminders in the home. In: Proceedings of the CHI 2011, BC, Canada (2011)Google Scholar
  17. 17.
    Walker, B. N., Lindsay, J., Nance, A., Nakano, Y., Palladino, D. K., Dingler, T., Jeon, M.: Spearcons (speech-based earcons) improve navigation performance in advanced auditory menus. Human Factors (2012) (Published online July 2, 2012 Print edition pending) Google Scholar
  18. 18.
    Walker, B.N., Kogan, A.: Spearcon performance and preference for auditory menus on a mobile phone. In: Stephanidis, C. (ed.) UAHCI 2009, Part II. LNCS, vol. 5615, pp. 445–454. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Jeon, M., Davison, B.K., Nees, M.A., Wilson, J., Walker, B.N.: Enhanced auditory menu cues improve dual task performance and are preferred with in-vehicle technologies. In: Proceedings of the AutomotiveUI 2009, Essen, Germany (2009)Google Scholar
  20. 20.
    Vicente, K.J.: Ecological interface design: Supporting operator adaptation, continuous learning, distributed collaborative work. In: Proceedings of the HCPC (1999)Google Scholar
  21. 21.
    Rasmussen, J.: The role of hierarchical knowledge representation in decision making and system management. IEEE Transactions on Systems, Man and Cybernetics 15, 234–243 (1985)CrossRefGoogle Scholar
  22. 22.
    Sanderson, P.M., Anderson, J., Watson, M.: Extending ecological interface design to auditory displays. CSIRO (2000)Google Scholar
  23. 23.
    Walker, B.N., Kramer, G.: Ecological psychoacoustics and auditory displays: Hearing, grouping, and meaning making. In: Neuhoff, J. (ed.) Ecological psychoacoustics, pp. 150–175. Academic Press, New York (2004)Google Scholar
  24. 24.
    Jeon, M.: Two or three things you need to know about AUI design or designers. In: Proceedings of the ICAD (2010)Google Scholar
  25. 25.
    Kurakata, K., Mizunami, T., Yomogida, H.: Guidelines on the temporal patterns of auditory signals for electronics home appliances: Report of the association for electric home appliances. Acoust. Sci. & Tech. 29(2), 176–184 (2008)CrossRefGoogle Scholar
  26. 26.
    Lee, J.-H., Jeon, M., Han, K.H.: Developing the design guideline of auditory user interface for domestic appliances. Korean Journal of the Science of Emotion & Sensibility 10(3), 307–320 (2007)Google Scholar
  27. 27.
    Kortum, P., Peres, S.C., Stallmann, K.: Mental workload measures of auditory stimuli heard during periods of waiting. In: Proceedings of the HFES (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Myounghoon Jeon
    • 1
  • Ju-Hwan Lee
    • 2
  1. 1.Michigan Technological UniversityHoughtonUSA
  2. 2.Korean German Institute of TechnologySeoulSouth Korea

Personalised recommendations