Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Computer Music Modeling and Retrieval

International Conference on Auditory Display

CMMR 2009, ICAD 2009: Auditory Display pp 306–329Cite as

From Signal to Substance and Back: Insights from Environmental Sound Research to Auditory Display Design

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 5954))

Abstract

A persistent concern in the field of auditory display design has been how to effectively use environmental sounds, which are naturally occurring familiar non-speech, non-musical sounds. Environmental sounds represent physical events in the everyday world, and thus they have a semantic content that enables learning and recognition. However, unless used appropriately, their functions in auditory displays may cause problems. One of the main considerations in using environmental sounds as auditory icons is how to ensure the identifiability of the sound sources. The identifiability of an auditory icon depends on both the intrinsic acoustic properties of the sound it represents, and on the semantic fit of the sound to its context, i.e., whether the context is one in which the sound naturally occurs or would be unlikely to occur. Relatively recent research has yielded some insights into both of these factors. A second major consideration is how to use the source properties to represent events in the auditory display. This entails parameterizing the environmental sounds so the acoustics will both relate to source properties familiar to the user and convey meaningful new information to the user. Finally, particular considerations come into play when designing auditory displays for special populations, such as hearing impaired listeners who may not have access to all the acoustic information available to a normal hearing listener, or to elderly or other individuals whose cognitive resources may be diminished. Some guidelines for designing displays for these populations will be outlined.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Gygi, B.: Studying environmental sounds the Watson way. J. Acoust. Soc. Am. 115(5), 2574 (2004)

    Google Scholar 

  2. Mynatt, E.: Designing with auditory icons. In: Proceedings of the 2nd International Conference on Auditory Display (ICAD 1994), Santa Fe, NM, U.S, pp. 109–119 (1994)

    Google Scholar 

  3. Zhao, F., Stephens, S.D.G., Sim, S.W., Meredith, R.: The use of qualitative questionnaires in patients having and being considered for cochlear implants. Clin. Otolaryngol. 22, 254–259 (1997)

    Article  Google Scholar 

  4. Warren, R.M.: Auditory perception and speech evolution. Annals of the New York Academy of Sciences, Origins and Evolution of Language 280, 708–717 (1976)

    Article  Google Scholar 

  5. Gaver, W.W.: Auditory icons: using sound in computer interfaces. Hum. Comput. Interact. 2(2), 167–177 (1986)

    Article  Google Scholar 

  6. Gaver, W.W.: The SonicFinder: An Interface that Uses Auditory Icons. Hum. Comput. Interact. 4(1), 67–94 (1989)

    Article  MathSciNet  Google Scholar 

  7. Kunkler-Peck, A.J., Turvey, M.T.: Hearing shape. J. Exp. Psychol. Human 26(1), 279–294 (2000)

    Article  Google Scholar 

  8. Carello, C., Anderson, K.L., Kunkler-Peck, A.J.: Perception of object length by sound. Psychol. Sci. 9(3), 211–214 (1998)

    Article  Google Scholar 

  9. Li, X., Logan, R.J., Pastore, R.E.: Perception of acoustic source characteristics: Walking sounds. J. Acoust. Soc. Am. 90(6), 3036–3049 (1991)

    Article  Google Scholar 

  10. Keller, P., Stevens, C.: Meaning From Environmental Sounds: Types of Signal-Referent Relations and Their Effect on Recognizing Auditory Icons. J. Exp. Psychol.-Appl. 10(1), 3–12 (2004)

    Article  Google Scholar 

  11. Absar, R., Guastavino, C.: Usability of Non-Speech Sounds in User Interfaces. In: Proceedings of the 14th International Conference on Auditory Display Paris, France, pp. 1–8 (2008)

    Google Scholar 

  12. Fernstroem, M., Brazil, E., Bannon, L.: HCI Design and Interactive Sonification for Fingers and Ears. IEEE MultiMedia 12(2), 36–44 (2005)

    Article  Google Scholar 

  13. Watson, C.S., Kidd, G.R.: Factors in the design of effective auditory displays. In: Proceedings of the 2nd International Conference on Auditory Display (ICAD 1994), Santa Fe, NM, U.S, pp. 293–303 (1994)

    Google Scholar 

  14. Mynatt, E.: Designing with auditory icons. In: Proceedings of the 2nd International Conference on Auditory Display (ICAD 1994), pp. 109–119. International Community for Auditory Display, Santa Fe, NM, U.S (1994)

    Google Scholar 

  15. Gaver, W.W.: Using and creating auditory icons. In: Kramer, G. (ed.) SFI studies in the sciences of complexity, pp. 417–446. Addison Wesley, Longman (1992)

    Google Scholar 

  16. Ballas, J.: Delivery of information through sound. In: Kramer, G. (ed.) SFI studies in the sciences of complexity, pp. 79–94. Addison Wesley, Longman (1992)

    Google Scholar 

  17. Ballas, J.A.: What is that sound? Some implications for sound design. In: Design Sonore, Paris, pp. 1–12 (2002)

    Google Scholar 

  18. Lucas, P.A.: An evaluation of the communicative ability of auditory icons and earcons. In: Proceedings of the 2nd International Conference on Auditory Display (ICAD 1994), Santa Fe, NM, U.S, pp. 121–128 (1994)

    Google Scholar 

  19. Lass, N.J., Eastman, S.K., Parrish, W.C., Ralph, D.: Listeners’ identification of environmental sounds. Percept. Motor. Skill. 55(1), 75–78 (1982)

    Google Scholar 

  20. Miller, J.D., Tanis, D.C.: Recognition memory for common sounds. Psychon. Sci. 23(4), 307–308 (1973)

    Google Scholar 

  21. Lawrence, D.M., Banks, W.P.: Accuracy of recognition memory for common sounds. Bull. Psychonom. Soc. 1(5A), 298–300 (1973)

    Google Scholar 

  22. Vanderveer, N.J.: Ecological acoustics: Human perception of environmental sounds. Dissertation Abstracts International 40(9-B), 4543 (1980)

    Google Scholar 

  23. Ballas, J.A.: Common factors in the identification of an assortment of brief everyday sounds. J. Exp. Psychol. Human 19(2), 250–267 (1993)

    Article  Google Scholar 

  24. Fabiani, M., Kazmerski, V.A., Cycowicz, Y.M.: Naming norms for brief environmental sounds: Effects of age and dementia. Psychophys. 33(4), 462–475 (1996)

    Article  Google Scholar 

  25. Gygi, B., Kidd, G.R., Watson, C.S.: Spectral-temporal factors in the identification of environmental sounds. J. Acoust. Soc. Am. 115(3), 1252–1265 (2004)

    Article  Google Scholar 

  26. Marcell, M.M., Borella, D., Greene, M., Kerr, E., Rogers, S.: Confrontation naming of environmental sounds. J. Clin. Exp. Neuropsyc. 22(6), 830–864 (2000)

    Article  Google Scholar 

  27. Myers, L.L., Letowski, T.R., Abouchacra, K.S., Kalb, J.T., Haas, E.C.: Detection and recognition of octave-band sound effects. J Am. Acad. Otolayrn 7, 346–357 (1996)

    Google Scholar 

  28. Shafiro, V.: Identification of environmental sounds with varying spectral resolution. Ear. Hear. 29(3), 401–420 (2008)

    Article  Google Scholar 

  29. Shafiro, V., Gygi, B., Cheng, M.-Y., Mulvey, M., Holmes, B.: Perception of speech and environmental sounds in cochlear implant patients. J. Acoust. Soc. Am. 123(5), 3303 (2008)

    Article  Google Scholar 

  30. Guillaume, A., Pellieux, L., Chastres, V., Blancard, C.: How long does it take to identify everyday sounds. In: ICAD 2004 -Tenth Meeting of the International Conference on Auditory Display, Sydney, Australia, pp. ICAD04-1–ICAD04-4 (2004)

    Google Scholar 

  31. Gygi, B.: From acoustics to perception: How to listen to meaningful sounds in a meaningful way. J. Acoust. Soc. Am. 113(4), 2326 (2003)

    Google Scholar 

  32. Warren, W.H., Verbrugge, R.R.: Auditory perception of breaking and bouncing events: A case study in ecological acoustics. J. Exp. Psychol. Human. 10(5), 704–712 (1984)

    Article  Google Scholar 

  33. Gaver, W.W.: What in the world do we hear? An ecological approach to auditory event perception. Ecol. Psychol. 5(1), 1–29 (1993)

    Article  MathSciNet  Google Scholar 

  34. Mynatt, E.D.: Transforming graphical interfaces into auditory interfaces for blind users. Hum. Comput. Interact. 12(1), 7–45 (1997)

    Article  Google Scholar 

  35. Bilger, R.C., Nuetzel, J.M., Rabinowitz, W.M., Rzeczkowski, C.: Standardization of a test of speech perception in noise. J. Speech. Hear. Res. 27, 32–48 (1984)

    Google Scholar 

  36. Hafter, E.R., Saberi, K.: A level of stimulus representation model for auditory detection and attention. J. Acoust. Soc. Am. 110(3), 1489 (2001)

    Article  Google Scholar 

  37. Schlauch, R.S., Hafter, E.R.: Listening bandwidths and frequency uncertainty in pure-tone signal detection. J. Acoust. Soc. Am. 90, 1332–1339 (1991)

    Article  Google Scholar 

  38. Watson, C.S., Foyle, D.C.: Central factors in the discrimination and identification of complex sounds. J. Acoust. Soc. Am. 78(1), 375–380 (1985)

    Article  Google Scholar 

  39. Ballas, J.A., Mullins, T.: Effects of context on the identification of everyday sounds. Hum. Perform. 4(3), 199–219 (1991)

    Article  Google Scholar 

  40. Gygi, B., Shafiro, V.: The Incongruency Advantage for Environmental Sounds Presented in Natural Auditory Scenes. Submitted (2008)

    Google Scholar 

  41. Gordon, R.D.: Attentional allocation during the perception of scenes. J. Exp. Psychol. Human. 30, 760–777 (2004)

    Article  Google Scholar 

  42. Lucas, P.A.: An evaluation of the communicative ability of auditory icons and earcons. In: Proceedings of the 2nd International Conference on Auditory Display (ICAD 1994), pp. 121–128. International Community for Auditory Display, Santa Fe, NM, U.S (1994)

    Google Scholar 

  43. Petrie, H., Morley, S.: The use of non-speech sounds in non-visual interfaces to the MS Windows GUI for blind computer users. In: Proceedings of the 5th International Conference on Auditory Display (ICAD 1998), pp. 1–5. University of Glasgow, U.K (1998)

    Google Scholar 

  44. Kidd, G.R., Watson, C.S.: The perceptual dimensionality of environmental sounds. Noise Cont. Eng. J. 51(4), 216–231 (2003)

    Article  Google Scholar 

  45. Gaver, W.W.: How do we hear in the world? Explorations in ecological acoustics. Ecol. Psychol. 5(4), 285–313 (1993)

    Article  MathSciNet  Google Scholar 

  46. William, W.G.: Auditory icons: using sound in computer interfaces. Hum. Comput. Interact. 2(2), 167–177 (1986)

    Article  MathSciNet  Google Scholar 

  47. Coleman, P.D.: An analysis of cues to auditory depth perception in free space. Psychological Bulletin 60(3), 302 (1963)

    Article  Google Scholar 

  48. Freed, D.: Auditory correlates of perceived mallet hardness for a set of recorded percussive sound events. J. Acoust. Soc. Am. 87(1), 311–322 (1990)

    Article  Google Scholar 

  49. Grey, J.M.: Multidimensional perceptual scaling of musical timbres. J. Acoust. Soc. Am. 61(5), 1270–1277 (1977)

    Article  Google Scholar 

  50. Lakatos, S., McAdams, S., Caussé, R.: The representation of auditory source characteristics: Simple geometric form. Percept. Psychophys. 59(8), 1180–1190 (1997)

    Google Scholar 

  51. Gygi, B., Kidd, G.R., Watson, C.S.: Similarity and Categorization of Environmental Sounds. Percept. Psychophys. 69(6), 839–855 (2007)

    Google Scholar 

  52. Aramaki, M., Kronland-Martinet, R.: Analysis-synthesis of impact sounds by real-time dynamic filtering. IEEE T Speech Audi P 14(2), 1–9 (2006)

    Article  Google Scholar 

  53. Avanzini, F.: Synthesis of Environmental Sounds in Interactive Multimodal Systems. In: Proceedings of the 13th International Conference on Auditory Display (ICAD 2007), Montreal, Canada, pp. 181–188 (2007)

    Google Scholar 

  54. Lee, J.-F., Shen, I.Y., Crouch, J., Aviles, W., Zeltzer, D., Durlach, N.: Using physically based models for collision-sound synthesis in virtual environments. J. Acoust. Soc. Am. 95(5), 2967 (1994)

    Article  Google Scholar 

  55. Cook, P.R.: Physically inspired sonic modeling (PhISM): Synthesis of percussive sounds. Comput. Music J. 21, 38–49 (1997)

    Article  Google Scholar 

  56. Rath, M.: An expressive real-time sound model of rolling. In: The 6th International Conference on Digital Audio Effects (DAFX 2003), pp. 165–168. University of London, Queen Mary (2003)

    Google Scholar 

  57. Stoelinga, C., Chaigne, A.: Time-Domain Modeling and Simulation of Rolling Objects. Acta Acust United Ac 93, 290–304 (2007)

    Google Scholar 

  58. van den Doel, K.: Physically-Based Models for Liquid Sounds. In: Tenth Meeting of the International Conference on Auditory Display, Sydney, Australia, pp. 1–8 (2004)

    Google Scholar 

  59. Avanzini, F., Serafin, S., Rocchesso, D.: Modeling Interactions Between Rubbed Dry Surfaces Using an Elasto-Plastic Friction Model. In: Proceedings of the COST-G6 Conf. Digital Audio Effects (DAFX 2002), Hamburg, pp. 111–116 (2002)

    Google Scholar 

  60. Lakatos, S., Cook, P.C., Scavone, G.P.: Selective attention to the parameters of a physically informed sonic model. J. Acoust. Soc. Am. 107(5,Pt1), L31-L36 (2000)

    Article  Google Scholar 

  61. Lutfi, R.A., Liu, C.-J.: Individual differences in source identification from synthesized impact sounds. J. Acoust. Soc. Am. 122(2), 1017–1028 (2007)

    Article  Google Scholar 

  62. Reed, R.K., Kidd, G.R.: Detection of Spectral Changes in Everyday Sounds. Indiana University, Unpublished data (2007)

    Google Scholar 

  63. Fernström, M., Brazil, E.: Human-Computer Interaction Design based on Interactive Sonification - Hearing Actions or Instruments/Agents. In: Proceedings of the 2004 International Workshop on Interactive Sonification, pp. 1–4. Bielefeld University, Germany (2004)

    Google Scholar 

  64. Heller, L.M.W.: When sound effects are better than the real thing. J. Acoust. Soc. Am. 111(5 pt.2), 2339 (2002)

    Google Scholar 

  65. Moore, B.C.J.: Cochlear hearing loss, pp. 47–88. Whurr Publishers, London (1998)

    Google Scholar 

  66. Badran, S., Osama, E.L.: Speech and environmental sound perception difficulties by patients with hearing loss requiring and using hearing aid. Indian J. Oto. 4(1), 13–16 (1998)

    Google Scholar 

  67. Shannon, R.V., Zeng, F.-G., Kamath, V., Wygonski, J., et al.: Speech recognition with primarily temporal cues. Science 270(5234), 303–304 (1995)

    Article  Google Scholar 

  68. Reed, C.M., Delhorne, L.A.: Reception of Environmental Sounds Through Cochlear Implants. Ear. Hear. 26(1), 48–61 (2005)

    Article  Google Scholar 

  69. Inverso, D.: Cochlear Implant-Mediated Perception of Nonlinguistic Sounds, Unpublished Thesis, Gallaudet University (2008)

    Google Scholar 

  70. Bronkhorst, A.W., Plomp, R.: Effect of multiple speechlike maskers on binaural speech recognition in normal and impaired hearing. J. Acoust. Soc. Am. 92(6), 3132 (1992)

    Article  Google Scholar 

  71. Festen, J.M., Plomp, R.: Effects of fluctuating noise and interfering speech on the speech-reception threshold for impaired and normal hearing. J. Acoust. Soc. Am. 88(4), 1725 (1990)

    Article  Google Scholar 

  72. Loizou, P.C., Hu, Y., Litovsky, R., Yu, G., Peters, R., Lake, J., Roland, P.: Speech recognition by bilateral cochlear implant users in a cocktail-party setting. J. Acoust. Soc. Am. 125(1), 372 (2009)

    Article  Google Scholar 

  73. Cherry, C.: Some experiments on the recognition of speech with one and with two ears. J. Acoust. Soc. Am. 26, 975–979 (1953)

    Article  Google Scholar 

  74. Salthouse, T.A.: The processing-speed theory of adult age differences in cognition. Psychol. Rev. 103(3), 403–428 (1996)

    Article  Google Scholar 

  75. Divenyi, P.L., Stark, P.B., Haupt, K.: Decline of Speech Understanding and Auditory Thresholds in the Elderly. J. Acoust. Soc. Am. 118, 1089–1100 (2005)

    Article  Google Scholar 

  76. Humes, L.E., Lee, J.H., Coughlin, M.P.: Auditory measures of selective and divided attention in young and older adults using single-talker competition. J. Acoust. Soc. Am. 120(5), 2926 (2006)

    Article  Google Scholar 

  77. Mynatt, E.D.: Transforming graphical interfaces into auditory interfaces for blind users. Hum. Comput. Interact. 12(1), 7–45 (1997)

    Article  Google Scholar 

  78. Roth, P., Petrucci, L., Pun, T., Assimacopoulos, A.: Auditory browser for blind and visually impaired users. In: CHI 1999 extended abstracts on Human factors in computing systems, pp. 1–2. ACM, Pittsburgh (1999)

    Google Scholar 

  79. Cobb, N.J., Lawrence, D.M., Nelson, N.D.: Report on blind subjects’ tactile and auditory recognition for environmental stimuli. Percept Mot Skills 48(2), 363–366 (1979)

    Google Scholar 

  80. Petrie, H., Morley, S.: The use of non-speech sounds in non-visual interfaces to the MS Windows GUI for blind computer users. In: Proceedings of the 5th International Conference on Auditory Display (ICAD 1998), pp. 1–5. British Computer Society, University of Glasgow, U.K (1998)

    Google Scholar 

  81. Wersényi, G.: Evaluation of auditory representations for selected applications of a graphical user interface. In: Proceedings of the 15th International Conference on Auditory Display (ICAD 2009), Re: New – Digital Arts Forum, Copenhagen, Denmark, pp. 41–48 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Veterans Affairs Northern California Health Care System, Martinez, CA, USA, 94553

    Brian Gygi

  2. Communications Disorders and Sciences, Rush University Medical Center,  

    Valeriy Shafiro

Authors
  1. Brian Gygi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valeriy Shafiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CNRS - LMA, 31 Chemin Joseph Aiguier, CNRS-LMA, 31 Chemin Joseph Aiguier, 13402, Marseille Cedex 20, France

    Sølvi Ystad

  2. Université Aix-Marseille, 38 bd. Charles Livon, Marseille, France

    Mitsuko Aramaki

  3. CNRS-LMA, 31, Chemin Joseph Aiguier, 13402, Marseille Cedex 20, France

    Richard Kronland-Martinet

  4. Aalborg University Esbjerg, Niels Bohr Vej 8, 6700, Esbjerg, Denmark

    Kristoffer Jensen

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gygi, B., Shafiro, V. (2010). From Signal to Substance and Back: Insights from Environmental Sound Research to Auditory Display Design. In: Ystad, S., Aramaki, M., Kronland-Martinet, R., Jensen, K. (eds) Auditory Display. CMMR ICAD 2009 2009. Lecture Notes in Computer Science, vol 5954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12439-6_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12439-6_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12438-9

  • Online ISBN: 978-3-642-12439-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation