Skip to main content

On the validity of virtual reality-based auditory experiments: a case study about ratings of the overall listening experience

Abstract

In recent years, new developments have led to an increasing number of virtual reality (VR)-based experiments, but little is known about their validity compared to real-world experiments. To this end, an experiment was carried out which compares responses given in a real-world environment to responses given in a VR environment. In the experiment, thirty participants rated the overall listening experience of music excerpts while sitting in a cinema and a listening booth being in a real-world environment and in a VR environment. In addition, the VR system that was used to carry out the sessions in the VR environment is presented in detail. Results indicate that there are only minor statistically significant differences between the two environments when the overall listening experience is rated. Furthermore, in the real-world environment, the ratings given in the listening booth were slightly higher than in the cinema.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    In this paper, the term real-world experiment is used to describe experiments that were conducted under laboratory conditions in the real-world (and not in the VR). Sometimes this term is also used as a synonym for the term field experiment.

  2. 2.

    M = mean, SD = standard deviation, N = number of samples.

  3. 3.

    The significance level \(\alpha\) is set to 0.05 in this paper.

References

  1. Agresti A (2002) Categorial data analysis, 2nd edn. Wiley, New York. ISBN: 0-471-36093-7

    Book  Google Scholar 

  2. Astheimer P (1993) What you see is what you hear-acoustics applied in virtual worlds. In: Proceedings of the IEEE 1993 symposium on research frontiers in virtual reality, pp 100–107. ISBN: 0-8186-4910-0

  3. Bella F (2004) Driving simulation in virtual reality for work zone design on highway: a validation study. In: The second SIIV international congress, Florence, Italy

  4. Berkhout AJ, de Vries D, Vogel P (1993) Acoustic control by wave field synthesis. J Acoust Soc Am 93(5):2764–2778

    Article  Google Scholar 

  5. Blauert J (1997) Spacial hearing. The psychophysics of human sound localization. The MIT Press, Cambridge. ISBN: 978-262-02413-6

  6. Blauert J, Jekosch U (2012) A layer model of sound quality. J Audio Eng Soc 60(1/2):4–12

    Google Scholar 

  7. Bossard C, Kermarrec G, Buche C, Tisseau J (2008) Transfer of learning in virtual environments: a new challenge? Virtual Real 12(3):151–161

    Article  Google Scholar 

  8. Bowman DA, McMahan RP (2007) Virtual reality: how much immersion is enough? Computer 40(7):36–43

    Article  Google Scholar 

  9. Cakmakci O, Rolland J (2006) Head-worn displays: a review. Journal of Disp Technol 2(3):199–216

    Article  Google Scholar 

  10. Cohen J (1988) Statistical power analysis for the behavioral sciences. Lawrence Erlbaum, Hillsdale. ISBN: 978-080-580283-2

    Google Scholar 

  11. Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC (1992) The CAVE: audio visual experience automatic virtual environment. Commun ACM 35(6):64–72

    Article  Google Scholar 

  12. DeFanti TA, Dawe G, Sandin DJ, Schulze JP, Otto P, Girado J, Kuester F, Smarr L, Rao R (2009) The StarCAVE, a third-generation CAVE and virtual reality optiportal. Futur Gener Comput Syst 25(2):169–178

    Article  Google Scholar 

  13. European Broadcasting Union (2011) Practical guidelines for production and implementation in accordance with EBU R 128 (version 2.0). European Broadcasting Union, Geneva

  14. Frechaud V (2013) Gui3D, v. 1.11

  15. Fritz CO, Morris PE, Richler JJ (2012) Effect size estimates: current use, calculations, and interpretation. J Exp Psychol Gen 141(1):2–18

    Article  Google Scholar 

  16. Garcia G (2002) Optimal filter partition for efficient convolution with short input/output delay. In: Proceedings of the AES 113th convention

  17. Gardner WG (1994) Efficient convolution without input/output delay. J. Audio Eng Soc 127–136 (preprint 3897)

  18. Gilkey R, Anderson T (2014) Binaural and spatial hearing in real and virtual environments. Taylor & Francis, London. ISBN: 978-131-778026-7

    Google Scholar 

  19. Gorzel M, Corrigan D, Kearney G, Squires J, Boland F (2012) Distance perception in virtual audio-visual environment. In: 25th UK conference of the audio engineering society: spatial audio in today’s 3D world, York, UK

  20. Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR (2008) Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg 95(9):1088–1097

    Article  Google Scholar 

  21. Hess W, Weishäupl J (2014) Replication of human head movements in 3 dimensions by a mechanical joint. In: Proceedings of the international conference on spatial audio (ICSA)

  22. Kozak JJ, Hancock PA, Arthur EJ, Chrysler ST (1993) Transfer of training from virtual reality. Ergonomics 36(7):777–784

    Article  Google Scholar 

  23. Kuhlen T, Assenmacher I, Lentz T (2007) A true spatial sound system for CAVE-like displays using four loudspeakers. In: Shumaker R (ed) Virtual reality, vol 4563. Springer, Berlin, pp 270–279. ISBN: 978-354-073334-8

    Chapter  Google Scholar 

  24. Larsson P, Västfjäll D, Kleiner M (2004) Perception of self-motion and presence in auditory virtual environments. In: Proceedings of the presence, pp 252–258

  25. Lathi BP, Green RA (2014) Essentials of digital signal processing. Cambridge University Press, Cambridge. ISBN: 978-110-705932-0

    Google Scholar 

  26. Le Callet P, Möller S, Perkis A (2012) Qualinet white paper on definitions of quality of experience (version 1.1). Qualinet, Dagstuhl

  27. Lindau A, Maempel H, Weinzierl S (2008) Minimum BRIR grid resolution for dynamic binaural synthesis. J Acoust Soc Am 123(5):3498–3498

    Article  Google Scholar 

  28. Loomis JM, Blascovich JJ, Beall AC (1999) Immersive virtual environment technology as a basic research tool in psychology. Behav Res Methods Instrum Comput 31(4):557–564

    Article  Google Scholar 

  29. Mershon DH, Desaulniers DH, Kiefer SA Jr, Amerson TL, Mills JT (1981) Perceived loudness and visually-determined auditory distance. Perception 10(5):531–543

    Article  Google Scholar 

  30. Müller S, Massarani P (2001) Transfer-function measurement with sweeps. J Audio Eng Soc 49(6):443–471

    Google Scholar 

  31. Novo P (2005) Auditory virtual environments. In: Blauert J (ed) Communication acoustics. Springer, Berlin, pp. 277–297

    Chapter  Google Scholar 

  32. Palomäki KJ, Tiitinen H, Mäkinen V, May JC, Alku P (2005) Spatial processing in human auditory cortex: the effects of 3D, ITD, and ILD stimulation techniques. Cogn Brain Res 24(3):364–379

    Article  Google Scholar 

  33. Pearson JL, Dollinger SJ (2004) Music preference correlates of jungian types. Personal Individ Differ 36(5):1005–1008

    Article  Google Scholar 

  34. Prince WF (1972) A paradigm for research on music listening. J Res Music Educ 20(4):445–455

    Article  Google Scholar 

  35. Psotka J (1995) Immersive training systems: virtual reality and education and training. Instr Sci 23(5–6):405–431

    Article  Google Scholar 

  36. Pysiewicz A (2014) On the validity of web-based auditory perception experiments. Master’s thesis, TU Berlin

  37. Rose FD, Attree EA, Brooks BM, Parslow DM, Penn PR, Ambihaipahan N (2000) Training in virtual environments: transfer to real world tasks and equivalence to real task training. Ergonomics 43(4):494–511

    Article  Google Scholar 

  38. Rumsey F, Zielinski S, Kassier R, Bech S (2005) Relationships between experienced listener ratings of multichannel audio quality and naive listener preferences. J Acoust Soc Am 117(6):3832–3840

    Article  Google Scholar 

  39. Sanchez-Vives MV, Slater M (2005) From presence to consciousness through virtual reality. Nat Rev Neurosci 6(6):332–339

    Article  Google Scholar 

  40. Sandel TT, Teas DC, Feddersen WE, Jeffress LA (1955) Localization of sound from single and paired sources. J Acoust Soc Am 27:842–852

    Article  Google Scholar 

  41. Schoeffler M, Herre J (2013) About the impact of audio quality on overall listening experience. In: Proceedings of the sound and music computing conference, Stockholm, Sweden, pp 48–53

  42. Schoeffler M, Herre J (2014a) Towards a listener model for predicting the overall listening experience. In: Proceedings of the audiomostly, Aalborg, Denmark

  43. Schoeffler M, Herre J (2014b) About the different types of listeners for rating the overall listening experience. In: Proceedings of the sound and music computing conference, Athens, Greece

  44. Schoeffler M, Hess W (2012) A comparison of highly configurable CPU- and GPU-based convolution engines. In: Audio engineering society convention no. 133, San Francisco, CA, USA

  45. Schoeffler M, Edler B, Herre J (2013a) How much does audio quality influence ratings of overall listening experience? In: Proceedings of the 10th international symposium on computer music multidisciplinary research (CMMR), pp 678–693, Marseille, France

  46. Schoeffler M, Stöter F-R, Bayerlein H, Edler B, Herre J (2013b) An experiment about estimating the number of instruments in polyphonic music: a comparison between internet and laboratory results. In: Proceedings of the 14th international society for music information retrieval conference, Curitiba, Brazil

  47. Schoeffler M, Adami A, Herre J (2014a) The influence of up- and down-mixes on the overall listening experience. In: Proceedings of the AES 137th convention, Los Angeles, CA, USA (preprint 9140)

  48. Schoeffler M, Conrad S, Herre J (2014b) The influence of the single/multi-channel-system on the overall listening experience. In: Proceedings of the AES 55th conference on spatial audio, Helsinki, Finland

  49. Schoeffler M, Stöter F, Edler B, Herre J (2015) Towards the next generation of web-based experiments: a case study assessing basic audio quality following the itu-r recommendation bs.1534 (MUSHRA). In: 1st web audio conference, Paris, France

  50. Schröder D, Wefers F, Pelzer S, Rausch D, Vorländer M, Kuhlen T (2010) Virtual reality system at RWTH Aachen University. In: Proceedings of the international symposium on room acoustics, Sydney, NSW, Australia. Australian Acoustical Society, NSW Division

  51. Schuemie MJ, van der Straaten P, Krijn M, van der Mast CA (2001) Research on presence in virtual reality: a survey. Cyberpsychol Behav Soc Netw 4(2):183–201

    Article  Google Scholar 

  52. Seeber BU, Fastl H (2004) On auditory-visual interaction in real and virtual environments. In: Proceedings of the 18th international congress on acoustics, pp 2293–2296, Kyoto, Japan

  53. Silzle A, Strauss H, Novo P (2004) IKA-SIM: a system to generate auditory virtual environments. In: Audio engineering society convention no. 116 (preprint 6016)

  54. Stanney K (1995) Realizing the full potential of virtual reality: human factors issues that could stand in the way. In: Proceedings the virtual reality annual international symposium ’95. IEEE Computer Society Press, Los Alamitos, pp 28–34

  55. Stanney KM, Mourant RR, Kennedy RS, Literature AROT (1998) Human factors issues in virtual environments: a review of the literature. Presence 7:327–351

    Article  Google Scholar 

  56. Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42(4):73–93

    Article  Google Scholar 

  57. Stockham Jr TG (1966) High-speed convolution and correlation. In: Proceedings of the spring joint computer conference, April 26–28, pp 229–233. ACM, New York, NY, USA

  58. Sveistrup H, McComas J, Thornton M, Marshall S, Finestone H, McCormick A, Babulic K, Mayhew A (2003) Experimental studies of virtual reality-delivered compared to conventional exercise programs for rehabilitation. Cyberpsychol Behav Soc Netw 6(3):245–249

    Article  Google Scholar 

  59. The OGRE Team (2013) OGRE game engine, v. 1.9.0

  60. Torger A, Farina A (2001) Real-time partitioned convolution for ambiophonics surround sound. In: IEEE workshop on applications of signal processing to audio and acoustics, pp 195–198

  61. Väljamäe A, Larsson P, Västfjäll D, Kleiner M (2004) Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments. In: Proceedings of the 7th annual workshop on presence

  62. van Dam A, Laidlaw DH, Simpson RM (2002) Experiments in immersive virtual reality for scientific visualization. Comput Graph 26(4):535–555

    Article  Google Scholar 

  63. Västfjäll D (2003) The subjective sense of presence, emotion recognition, and experienced emotions in auditory virtual environments. Cyberpsychol Behav 6(2):181–188

    Article  Google Scholar 

  64. Vora J, Nair S, Gramopadhye AK, Duchowski AT, Melloy BJ, Kanki B (2002) Using virtual reality technology for aircraft visual inspection training: presence and comparison studies. Appl Ergon 33(6):559–570

    Article  Google Scholar 

  65. Welch N, Krantz JH (1996) The world wide web as a medium for psychoacoustical demonstrations and experiments: experience and results. Behav Res Methods Instrum Comput 28(2):192–196

    Article  Google Scholar 

  66. Werner S, Siegel A (2011) Effects of binaural auralization via headphones on the perception of acoustic scenes. In: Proceedings of the 3rd international symposium on auditory and audiological research (ISAAR), Nyborg, Denmark

  67. Werner S, Liebetrau J, Sporer T (2012) Audio–visual discrepancy and the influence on vertical sound source localization. In: Proceedings of the 4th international workshop on quality of multimedia experience (QoMEX), pp 133–139, Melbourne, Australia

  68. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  69. Witmer BG, Bailey JH, Knerr BW, Parsons KC (1996) Virtual spaces and real world places: transfer of route knowledge. Int J Hum Comput Stud 45(4):413–428

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Alexander Adami for taking pictures of the experiment apparatus and Marlene Röß for representing a participant in the pictures.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael Schoeffler.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Schoeffler, M., Gernert, J.L., Neumayer, M. et al. On the validity of virtual reality-based auditory experiments: a case study about ratings of the overall listening experience. Virtual Reality 19, 181–200 (2015). https://doi.org/10.1007/s10055-015-0270-8

Download citation

Keywords

  • Virtual reality-based experiments
  • Overall listening experience
  • Convolution engine
  • Oculus Rift