Skip to main content
SpringerLink
Log in

Auditory-Induced Presence in Mixed Reality Environments and Related Technology

  • Chapter
  • First Online:
The Engineering of Mixed Reality Systems

Part of the book series: Human-Computer Interaction Series ((HCIS))

Abstract

Presence, the “perceptual illusion of non-mediation,” is often a central goal in mediated and mixed environments, and sound is believed to be crucial for inducing high-presence experiences. This chapter provides a review of the state of the art within presence research related to auditory environments. Various sound parameters such as externalization and spaciousness and consistency within and across modalities are discussed in relation to their presence-inducing effects. Moreover, these parameters are related to the use of audio in mixed realities and example applications are discussed. Finally, we give an account of the technological possibilities and challenges within the area of presence-inducing sound rendering and presentation for mixed realities and outline future research aims.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Loomis, J.M., Klatzky, R.L., Golledge, R.G.: Auditory distance perception in real, virtual and mixed environments. In Y. Othta & H. Tamura (Eds.), Mixed reality: Merging real and virtual worlds. Ohmsha, Tokyo (1999)

    Google Scholar 

  2. Kyota, H., Takanobu, H., Asako, K., Fumihisa, S., Hideyuki, T.: Mixed reality system using audio and visual senses: Implementation of simultaneous presentation in both audio and visual MR. Proceedings of the Virtual Reality Society of Japan Annual Conference. 11, 2A2–3 (2006)

    Google Scholar 

  3. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, E77–D(2), 1321–1329 (1994)

    Google Scholar 

  4. Pörschmann, C.: One’s own voice in auditory virtual environments. Acta Acustica, 87, 378–388 (2001)

    Google Scholar 

  5. Nordahl, R.: Self-induced footsteps sounds in virtual reality: Latency, recognition, quality and presence. Proceedings of the Eight Annual International Workshop Presence, London, UK (2004)

    Google Scholar 

  6. Lombard, M., Ditton, T.B.: At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication, 13(3) (1997)

    Google Scholar 

  7. Gilkey, R.H., Weisenberger, J.M.: The sense of presence for the suddenly deafened adult: implications for virtual environments. Presence: Teleoperators and Virtual Environments, 4(4), 357–363 (1995)

    Google Scholar 

  8. Ijsselsteijn, W.A.: Presence in depth. Doctoral dissertation, Eindhoven University of Technology, Eindhoven, The Netherlands (2004)

    Google Scholar 

  9. Slater, M: A note on presence terminology. Presence-Connect, 3(3), 1–5 (2003)

    MathSciNet  Google Scholar 

  10. Pope, J., Chalmers, A.: Multi-sensory rendering: combining graphics and acoustics. Proceedings of the 7th International Conference in Central Europe on Computer Graphics, Czech Republic, 233–242 (1999)

    Google Scholar 

  11. Guttman, S.E., Gilroy, L.A., Blake, R.: Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychological Science, 16(3), 228–235 (2005)

    Article  Google Scholar 

  12. Kleiner, M., Dalenbäck, B-I., Svensson, P.: Auralization: an overview. Journal of the Audio Engineering Society, 41(11), 861–875 (1993)

    Google Scholar 

  13. Begault, D.R.: 3D sound for virtual reality and multimedia. Academic Press Professional, London (1994)

    Google Scholar 

  14. Rumsey, F.: Spatial audio. Focal Press Oxford; Boston (2001)

    Google Scholar 

  15. Shilling, R.D., Shinn-Cunningham, B.G.: Virtual auditory displays. In K.M. Stanney (Ed.), Handbook of virtual environments: Design, implementation, and applications (pp. 65–92). Lawrence Erlbaum Associates, London, Mahwah, NJ (2002)

    Google Scholar 

  16. Hughes, C.E., Stapleton, C.B., Hughes, D.E., Smith, E.M.: Mixed reality in education, entertainment, and training. IEEE Computer Graphics and Applications, 25(6), 24–30 (2005)

    Article  Google Scholar 

  17. Jarlengrip, J.: Object detection system and method. International Patent no. WO/2007/049995. (2007)

    Google Scholar 

  18. Zimmermann, R., Kyriakakis, C., Shahabi, C., Papadopolous, C., Sawchuck, A.A., Neumann, U.: The remote media immersion system. IEEE MultiMedia, 11(2), 48–57 (2004)

    Article  Google Scholar 

  19. Pulkki, V.: Virtual sound source positioning using vector base amplitude panning. Journal of the Audio Engineering Society, 45(6), 456–466 (1997)

    Google Scholar 

  20. Gerzon, M.A.: Ambisonics in multichannel broadcasting and video. Journal of the Audio Engineering Society, 33(11), 859–871 (1985)

    Google Scholar 

  21. Berkhout, A.J.: A holographic approach to acoustic control. Journal of the Audio Engineering Society, 36(12), 977–995 (1988)

    Google Scholar 

  22. Horbach, U., Corteel, E., Pellegrini, R.S., Hulsebos, E.: Real-time rendering of dynamic scenes using wave field synthesis. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’02), Lausanne, Switzerland, 1, 517–520. (2002)

    Google Scholar 

  23. Horbach, U., Boone, M.M.: Future transmission and rendering formats for multichannel sound. Proceedings of the AES 16th Conference on Spatial Sound Reproduction, Rovaniemi, Finland (1999)

    Google Scholar 

  24. Murray, C.D., Arnold, P., Thornton, B.: Presence accompanying induced hearing loss: implications for immersive virtual environments. Presence: Teleoperators and Virtual Environments, 9(2), 137–148 (2000)

    Article  Google Scholar 

  25. Blauert, J.: Spatial hearing, revised edition. The MIT Press, Cambridge, MA (1997)

    Google Scholar 

  26. Wenzel, E.M., Arruda, M., Kistler, D.J., Wightman, F.L.: Localization using nonindividualized head-related transfer functions. Journal of the Acoustical Society of America, 94(1), 111–123 (1993)

    Article  Google Scholar 

  27. Zotkin, D.N., Duraiswami, R., Davis, L.S.: Rendering localized spatial audio in a virtual auditory space. IEEE Transactions on Multimedia, 6(4), 553–564 (2004)

    Article  Google Scholar 

  28. Hofman, P.M., Riswick, J.G.A., van Opstal, A.J.: Relearning sound localization with new ears. Nature Neuroscience, 1(5), 417–421 (1998)

    Article  Google Scholar 

  29. Durlach, N.I., Shinn-Cunningham, B.G., Held, R.M.: Supernormal auditory localization. I. General background. Presence: Teleoperators and Virtual Environments, 2(2), 89–103 (1993)

    Google Scholar 

  30. Gupta, N., Barreto, A., Ordonez, C.: Spectral modification of head-related transfer functions for improved virtual sound spatialization. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Orlando, FL, 2, 1953–1956 (2002)

    Google Scholar 

  31. Härmä, A., Jakka, J., Tikander, M., Karjalainen, M., Lokki, T., Hiipakka, J., Lorho, G.: Augmented reality audio for mobile and wearable appliances. Journal of the Audio Engineering Society, 52(6), 618–639 (2004)

    Google Scholar 

  32. Evans, M.J., Tew, A.I., Angus, J.A.S.: Spatial audio teleconferencing: Which way is better?: Proceedings of the fourth International Conference on Auditory Display (ICAD ’97), Palo Alto, California, 29–37 (1997)

    Google Scholar 

  33. Loomis, J.M., Golledge, R.G., Klatzky, R.L.: Navigation system for the blind: auditory display modes and guidance. Presence: Teleoperators and Virtual Environments, 7(2), 193–203 (1998)

    Article  Google Scholar 

  34. Tonndorf, J.: Bone conduction. In J. Tobias (Ed.), Foundations of modern auditory theory (Vol. 2, pp. 197–237). Academic Press, New York (1972)

    Google Scholar 

  35. Stenfelt, S., Goode, R.L.: Bone conducted sound: physiological and clinical aspects. Otology & Neurotology, 26(6), 1245–1261, (2005)

    Article  Google Scholar 

  36. Pörschmann, C.: Influences of bone conduction and air conduction on the sound of one’s own voice. Acustica – Acta Acustica, 86(6), 1038–1045 (2000)

    Google Scholar 

  37. Snik, A.F.M., Bosman, A.J., Mylanus, E.A.M., Cremers, C.W.R.J.: Candidacy for the bone-anchored hearing aid. Audiology & Neurotology, 9(4), 190–196 (2004)

    Article  Google Scholar 

  38. Väljamäe, A., Tajadura-Jiménez, A., Larsson, P., Västfjäll, D., Kleiner M.: Binaural bone-conducted sound in virtual environments: evaluation of a portable, multimodal motion simulator prototype. Journal of Acoustic Science and Technology, 29(2), 149–155 (2008)

    Article  Google Scholar 

  39. Gaye, L., Mazé, R., Holmquist, L.E.: Sonic City: The urban environment as a musical interface. Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada, 109–115. (2003)

    Google Scholar 

  40. Sanders, R.D. Jr., Scorgie, M.A.: The effect of sound delivery methods on a user’s sense of presence in a virtual environment. Master thesis Naval Postgraduate School, Monterey, CA (2002)

    Google Scholar 

  41. Durlach, N.I., Mavor, A.S.: Virtual reality scientific and technological challenges. National Academy Press, Washington, DC (1995)

    Google Scholar 

  42. Savioja, L., Huopaniemi, J., Lokki, T., Väänänen, R.: Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society, 47(9), 675–705 (1999)

    Google Scholar 

  43. Jot, J.-M., Warusfel, O.: A real-time spatial sound processor for music and virtual reality applications. Proceedings of the 1995 International Computer Music Conference, Banff, AB, Canada, 294–295 (1995)

    Google Scholar 

  44. Faller, C., Baumgarte, F.: Binaural cue coding – Part II: schemes and applications. IEEE Transactions on Speech and Audio Processing, 11(6), 520–531. (2003)

    Article  Google Scholar 

  45. Breebaart, J., van de Par, V., Kohlrausch, A., Schuijers, K.: Parametric coding of stereo audio. EURASIP Journal on Applied Signal Processing, 9, 1305–1322 (2005)

    Google Scholar 

  46. Tsingos, N., Gallo, E., Drettakis, G.: Perceptual audio rendering of complex virtual environments. INRIA technical report RR-4734, Feb. 2003, REVES/INRIA Sophia-Antipolis (2003)

    Google Scholar 

  47. Ramsdell, D.A.: The psychology of the hard-of-hearing and deafened adult. In H. Davis & S.R. Silverman (Eds.), Hearing and deafness (4th ed., pp. 499–510). Holt, Rinehart & Winston, New York (1978)

    Google Scholar 

  48. Hendrix, C., Barfield, W.: The sense of presence within auditory virtual environments. Presence: Teleoperators and Virtual Environments, 5(3), 290–301 (1996)

    Google Scholar 

  49. Väljamäe, A., Larsson, P., Västfjäll, D., Kleiner, M.: Auditory presence, individualized head-related transfer functions and illusory ego-motion in virtual environments. Proceedings of the Seventh Annual International Workshop Presence, Valencia, Spain, 141–147 (2004)

    Google Scholar 

  50. Begault, D.R., Wenzel, E.M., Anderson, M.R.: Direct comparison of the impact of head-tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. Journal of the Audio Engineering Society, 49(10), 904–916 (2001)

    Google Scholar 

  51. Larsson, P., Västfjäll, D., Kleiner, M.: Auditory information consistency and room acoustic cues increase presence in virtual environments. Acoustical Science and Technology, 29(2), 191–194 (2008)

    Article  Google Scholar 

  52. Larsson, P., Västfjäll, D., Kleiner, M.: On the quality of experience: A multi-modal approach to perceptual ego-motion and sensed presence in virtual environments. Proceedings of the First ITRW on Auditory Quality of Systems, Akademie Mont-Cenis, Germany. (2003)

    Google Scholar 

  53. Ozawa, K., Chujo, Y., Suzuki, Y., Sone, T.: Psychological factors involved in auditory presence. Acoustical Science and Technology, 24(1), 42–44 (2003)

    Article  Google Scholar 

  54. Rumsey, F.: Spatial quality evaluation for reproduced sound: terminology, meaning, and a scene-based paradigm. Journal of the Audio Engineering Society, 50(9), 651–666 (2002)

    Google Scholar 

  55. Freeman, J., Lessiter, J.: Here there & everywhere: The effects of multi-channel audio on presence. Proceedings of The Seventh International Conference on Audio Display, Helsinki, Finland, 231–234 (2001)

    Google Scholar 

  56. Ozawa, K., Miyasaka, M.: Effects of reproduced sound pressure levels on auditory presence. Acoustical Science and Technology, 25(3), 207–209 (2004)

    Article  Google Scholar 

  57. Serafin, G., Serafin, S.: Sound design to enhance presence in photorealistic virtual reality. Proceedings of the 2004 International Conference on Auditory Display (ICAD ’04), Sydney, Australia (2004)

    Google Scholar 

  58. Chueng, P., Marsden, P.: Designing auditory spaces to support sense of place: The role of expectation. Position paper for The Role of Place in On-line Communities Workshop, CSCW2002, New Orleans (2002)

    Google Scholar 

  59. Larsson, P., Västfjäll, D., Olsson, P., Kleiner, M.: When what you see is what you hear: Auditory-visual integration and presence in virtual environments. Proceedings of the 10th Annual International Workshop Presence, 11–18. Barcelona, Spain (2007)

    Google Scholar 

  60. Ozawa, K., Ohtake, S., Suzuki, Y., Sone, T.: Effects of visual information on auditory presence. Acoustical Science and Technology, 24(2), 97–99 (2003)

    Article  Google Scholar 

  61. Knapp, J.M., Loomis, J.M.: Limited field of view of head-mounted displays is not the cause of distance underestimation in virtual environments. Presence: Teleoperators and Virtual Environments, 13(5), 572–577 (2004)

    Article  Google Scholar 

  62. Thompson, W.B., Willemsen, P., Gooch, A.A., Creem-Regehr, S.H., Loomis, J.M., Beall, A.C.: Does the quality of the computer graphics matter when judging distances in visually immersive environments? Presence: Teleoperators and Virtual Environments, 13(5), 560–571 (2004)

    Article  Google Scholar 

  63. 37Murch, W.: Dense clarity – Clear density. Paper delivered at the “Volume: Bed of sound” program/exhibition, PS1/MoMA, New York, NY. http://www.ps1.org/cut/volume/murch.html (2000)

  64. Neuhoff, J.G: Ecological psychoacoustics: Introduction and history. In J.G. Neuhoff (Ed.), Ecological psychoacoustics. (pp. 1–13). Elsevier Academic Press, Amsterdam, Boston (2004)

    Chapter  Google Scholar 

  65. Hughes, D.E., Thropp, J., Holmquist, J., Moshell, J.M.: Spatial perception and expectation: factors in acoustical awareness for MOUT training. Proceedings of Army Science Conference (ASC) 2004, Orlando, FL (2004)

    Google Scholar 

  66. Weiss, E., Belton J.: Film sound: Theory and practice. Columbia University Press, New York (1985)

    Google Scholar 

  67. Väljamäe, A., Tajadura-Jiménez, A., Larsson P., Västfjäll D., Kleiner M.: Handheld Experiences: Using Audio To Enhance the Illusion of Self-Motion, IEEE MultiMedia, vol. 15, no.4, pp. 68–75, 2008

    Google Scholar 

  68. Riecke B.E., Väljamäe, A., Schulte-Pelkum, J.: Moving sounds enhance the visually-induced self-motion illusion (circular vection) in Virtual Reality ACM Transactions on Applied Perception (TAP) Volume 6, Issue 2, Article No. 7 (2009)

    Google Scholar 

  69. Väljamäe, A., Soto-Faraco, S.: Filling-in visual motion with sounds. Acta Psychologica, 129(2), 249–254. (2008)

    Article  Google Scholar 

Download references

Acknowledgments

The work presented in this chapter was supported by the EU FET Presence Research Initiative project POEMS (IST-2001-39223), the EU FET Integrated project PRESENCCIA (Project Number 27731), and the Swedish Science Council (VR). The first author thanks the Swedish Foundation for Strategic Research for its support. The second author thanks Alfred Ots’ scholarship foundation. We would also like to thank Armin Kohlrausch for commenting on earlier drafts of this chapter.

Author information

Authors and Affiliations

  1. Volvo Technology Corporation, SE-405 08, Göteborg, Sweden

    Pontus Larsson

  2. Department of Applied Acoustics, Chalmers University of Technology, SE-412 96, Göteborg, Sweden

    Pontus Larsson, Aleksander Väljamäe, Daniel Västfjäll, Ana Tajadura-Jiménez & Mendel Kleiner

  3. Research Laboratory for Synthetic Perceptive, Emotive and Cognitive Systems (SPECS), Institute of Audiovisual Studies, Universitat Pompeu Fabra, Barcelona, Spain

    Aleksander Väljamäe

  4. Department of Psychology, Göteborg University, Göteborg, Sweden

    Daniel Västfjäll

Authors
  1. Pontus Larsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aleksander Väljamäe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Västfjäll
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ana Tajadura-Jiménez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mendel Kleiner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pontus Larsson .

Editor information

Editors and Affiliations

  1. Institut de Recherches en, Université Toulouse III, route de Narbonne 118 , Toulouse CX 4, 31062, France

    Emmanuel Dubois

  2. Dept. Computing Science, University of Glasgow, Lilybank Gdns. 17, Glasgow, G12 8QQ, United Kingdom

    Philip Gray

  3. Labo. d'Informatique de Grenoble (LIG), Université Grenoble I, rue de la Bibliotheque 385, Grenoble, 38041, France

    Laurence Nigay

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag London

About this chapter

Cite this chapter

Larsson, P., Väljamäe, A., Västfjäll, D., Tajadura-Jiménez, A., Kleiner, M. (2010). Auditory-Induced Presence in Mixed Reality Environments and Related Technology. In: Dubois, E., Gray , P., Nigay, L. (eds) The Engineering of Mixed Reality Systems. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84882-733-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-84882-733-2_8

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84882-732-5

  • Online ISBN: 978-1-84882-733-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation