Skip to main content
Log in

A Computational Model to Implement Binaural Synthesis in a Hard Real-Time Auditory Virtual Environment

  • Original Paper
  • Published:
Acoustics Australia Aims and scope Submit manuscript

Abstract

There is a growing interest in the development and the evaluation of real-time auditory virtual environments (AVE). The implementation of this type of simulation system in general purpose computers is a still a challenge, and there are few studies that evaluated the perceived quality of synthetized sounds of simulated acoustic scenes. To evoke in the listener a correct image of the modeling space, the system must be dynamic and interactive. That is, it must respond to the changes in the acoustic scenario produced by the listener movement, in a perceptually acceptable time and with an update rate that guarantees continuity in the reproduction of sound events. Hard real-time systems ensure that a given task runs within a given time interval, providing deterministic behavior for applications with time restrictions. In the current article, a computational model to implement binaural synthesis in a hard real-time AVE is presented and evaluated. The computer model was implemented in an open-source auralization system. Measurements and real-time simulations on a university classroom were carried out to perform a reverberation time parameters validation and a system performance evaluation. Also, measured and simulated binaural soundtracks (composed from anechoic stimuli) were compared in terms of three selected perceptual attributes for subjective evaluations of static positions. The results showed that real-time performance was acceptable according to values previously reported in the literature and that computer prediction errors for the measured parameters were within the subjective difference limens. The computational model was able to generate an AVE with an acceptable overall perceptual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://github.com/ftommasini/avrs.

References

  1. Kleiner, M., Dalenbäck, B.-I., Svensson, U.P.: Auralization—an overview. J. Audio Eng. Soc. 41, 861–875 (1993)

    Google Scholar 

  2. Savioja, L., Huopaniemi, J., Lokki, T., Väänänen, R.: Creating interactive virtual acoustic environments. J. Audio Eng. Soc. 47, 675–705 (1999)

    Google Scholar 

  3. Lokki, T., Pätynen, J., Tervo, S., Siltanen, S., Savioja, L.: Engaging concert hall acoustics is made up of temporal envelope preserving reflections. J. Acoust. Soc. Am. 129, EL223–EL228 (2011). https://doi.org/10.1121/1.3579145

    Article  Google Scholar 

  4. Bilbao, S., Hamilton, B.: Wave-based room acoustics simulation: explicit/implicit finite volume modeling of viscothermal losses and frequency-dependent boundaries. J. Audio Eng. Soc. 65, 78–89 (2017)

    Article  Google Scholar 

  5. Lentz, T., Schröder, D., Vorländer, M., Assenmacher, I.: Virtual reality system with integrated sound field simulation and reproduction. EURASIP J. Adv. Signal Process. 2007, 1–17 (2007)

    Article  Google Scholar 

  6. Yuan, Y., Fu, Z., Xu, M., Xie, L., Cong, Q.: Externalization improvement in a real-time binaural sound image rendering system. In: 2015 International Conference on Orange Technologies (ICOT), pp. 165–168. IEEE, Hong Kong, China (2015)

  7. Wenzel, E.M., Miller, J.D., Abel, J.S.: Sound Lab: A real-time, software-based system for the study of spatial hearing. In: AES 108th Convention, preprint 5140, Audio Engineering Society, Paris, France (2000)

  8. Scarpaci, J.W.: Creation of a system for real time virtual auditory space and its application to dynamic sound localization. Doctoral thesis, Boston University, Boston, MA (2006)

  9. Noisternig, M., Katz, B.F.G., Siltanen, S., Savioja, L.: Framework for real-time auralization in architectural acoustics. Acta Acust. United Acust. 94, 1000–1015 (2008). https://doi.org/10.3813/AAA.918116

    Article  Google Scholar 

  10. Geier, M., Ahrens, J., Spors, S.: The SoundScape Renderer: A unified spatial audio reproduction framework for arbitrary rendering methods. In: AES 124th Convention, paper 7330, Audio Engineering Society, Amsterdam, The Netherlands (2008)

  11. Blauert, J.: Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge (1997)

    Google Scholar 

  12. Vorländer, M.: Computer simulations in room acoustics: concepts and uncertainties. J. Acoust. Soc. Am. 133, 1203–1213 (2013). https://doi.org/10.1121/1.4788978

    Article  Google Scholar 

  13. Lindau, A., Weinzierl, S.: Assessing the plausibility of virtual acoustic environments. Acta Acust. United Acust. 98, 804–810 (2012). https://doi.org/10.3813/AAA.918562

    Article  Google Scholar 

  14. Pike, C., Melchior, F., Tew, T.: Assessing the plausibility of non-individualised dynamic binaural synthesis in a small room. In: AES 55th International Conference: Spatial Audio, paper 6–1, Audio Engineering Society, Helsinki, Finland (2014)

  15. Brinkmann, F., Lindau, A., Weinzierl, S.: On the authenticity of individual dynamic binaural synthesis. J. Acoust. Soc. Am. 142, 1784–1795 (2017). https://doi.org/10.1121/1.5005606

    Article  Google Scholar 

  16. Langendijk, E.H.A., Bronkhorst, A.W.: Fidelity of three-dimensional-sound reproduction using a virtual auditory display. J. Acoust. Soc. Am. 107, 528–537 (2000). https://doi.org/10.1121/1.428321

    Article  Google Scholar 

  17. Moore, A.H., Tew, A.I., Nicol, R.: An initial validation of individualized crosstalk cancellation filters for binaural perceptual experiments. J. Audio Eng. Soc. 58, 36–45 (2010)

    Google Scholar 

  18. Oberem, J., Masiero, B., Fels, J.: Experiments on authenticity and plausibility of binaural reproduction via headphones employing different recording methods. Appl. Acoust. 114, 71–78 (2016). https://doi.org/10.1016/j.apacoust.2016.07.009

    Article  Google Scholar 

  19. Lindau, A., Erbes, V., Lepa, S., Maempel, H.-J., Brinkman, F., Weinzierl, S.: A spatial audio quality inventory (SAQI). Acta Acust. United Acust. 100, 984–994 (2014). https://doi.org/10.3813/AAA.918778

    Article  Google Scholar 

  20. Lokki, T., Jarvelainen, H.: Subjective evaluation of auralization of physics-based room acoustics modeling. In: Proceedings of the 2001 International Conference on Auditory Display, Espoo, Finland (2001)

  21. Choi, Y.-J., Fricke, F.R.: A comparison of subjective assessments of recorded music and computer simulated auralizations in two auditoria. Acta Acust. United Acust. 92, 604–611 (2006)

    Google Scholar 

  22. Yang, W., Hodgson, M.: Validation of the auralization technique: comparative speech-intelligibility tests in real and virtual classrooms. Acta Acust. United Acust. 93, 991–999 (2007)

    Google Scholar 

  23. Postma, B.N.J., Katz, B.F.G.: Perceptive and objective evaluation of calibrated room acoustic simulation auralizations. J. Acoust. Soc. Am. 140, 4326–4337 (2016). https://doi.org/10.1121/1.4971422

    Article  Google Scholar 

  24. Tommasini, F.C., Ramos, O.A., Ferreyra, S., Guido, R.M.: Sistema de realidad acústica virtual en tiempo real: AVRS. In: Proceedings of IX Congreso Iberoamericano de Acústica (FIA 2014), Valdivia, Chile (2014)

  25. Mantegazza, P., Dozio, E.L., Papacharalambous, S.: RTAI: real time application interface. Linux J. 2000, 10 (2000)

    Google Scholar 

  26. Arm, J., Bradac, Z., Kaczmarczyk, V.: Real-time capabilities of Linux RTAI. IFAC Pap. 49, 401–406 (2016). https://doi.org/10.1016/j.ifacol.2016.12.080

    Article  Google Scholar 

  27. Allen, J.B., Berkley, D.A.: Image method for efficiently simulating small-room acoustics. J. Acoust. Soc. Am. 65, 943–950 (1979)

    Article  Google Scholar 

  28. Borish, J.: Extension of the image model to arbitrary polyhedra. J. Acoust. Soc. Am. 75, 1827–1836 (1984). https://doi.org/10.1121/1.390983

    Article  Google Scholar 

  29. Bradley, J.S., Sato, H., Picard, M.: On the importance of early reflections for speech in rooms. J. Acoust. Soc. Am. 113, 3233–3244 (2003). https://doi.org/10.1121/1.1570439

    Article  Google Scholar 

  30. Jot, J.-M.: Efficient models for reverberation and distance rendering in computer music and virtual audio reality. In: International Computer Music Conference Proceedings 1997 (1997)

  31. Schlecht, S.J., Habets, E.A.P.: Feedback delay networks: echo density and mixing time. IEEEACM Trans. Audio Speech Lang. Process. 25, 374–383 (2017). https://doi.org/10.1109/TASLP.2016.2635027

    Article  Google Scholar 

  32. Wendt, T., van de Par, S., Ewert, S.D.: A computationally-efficient and perceptually-plausible algorithm for binaural room impulse response simulation. J. Audio Eng. Soc. 62, 748–766 (2014)

    Article  Google Scholar 

  33. Dalenbäck, B.-I.L.: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. J. Acoust. Soc. Am. 100, 899–909 (1996). https://doi.org/10.1121/1.416249

    Article  Google Scholar 

  34. Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J.E., Pingali, G., Min, P., Ngan, A.: A beam tracing method for interactive architectural acoustics. J. Acoust. Soc. Am. 115, 739 (2004). https://doi.org/10.1121/1.1641020

    Article  Google Scholar 

  35. Vorländer, M.: Simulation of the transient and steady-state sound propagation in rooms using a new combined ray-tracing/image-source algorithm. J. Acoust. Soc. Am. 86, 172–178 (1989). https://doi.org/10.1121/1.398336

    Article  Google Scholar 

  36. Lehmann, E.A., Johansson, A.M.: Diffuse reverberation model for efficient image-source simulation of room impulse responses. IEEE Trans. Audio Speech Lang. Process. 18, 1429–1439 (2010). https://doi.org/10.1109/TASL.2009.2035038

    Article  Google Scholar 

  37. Lehmann, E.A., Johansson, A.M.: Prediction of energy decay in room impulse responses simulated with an image-source model. J. Acoust. Soc. Am. 124, 269–277 (2008). https://doi.org/10.1121/1.2936367

    Article  Google Scholar 

  38. Cremer, L., Müller, H.A.: Principles and applications of room acoustics. Appl. Sci. 1, (1982)

  39. Defrance, G., Polack, J.: Measuring the mixing time in auditoria. J. Acoust. Soc. Am. 123, 3499 (2008). https://doi.org/10.1121/1.2934368

    Article  Google Scholar 

  40. Hidaka, T., Yamada, Y., Nakagawa, T.: A new definition of boundary point between early reflections and late reverberation in room impulse responses. J. Acoust. Soc. Am. 122, 326–332 (2007). https://doi.org/10.1121/1.2743161

    Article  Google Scholar 

  41. Lindau, A., Kosanke, L., Weinzierl, S.: Perceptual evaluation of physical predictors of the mixing time in binaural room impulse responses. In: AES 128th Convention, paper 8089, Audio Engineering Society, London, UK (2010)

  42. Defrance, G., Polack, J.: Estimating the mixing time of concert halls using the eXtensible Fourier Transform. Appl. Acoust. 71, 777–792 (2010). https://doi.org/10.1016/j.apacoust.2010.05.011

    Article  Google Scholar 

  43. Grijalva, F., Martini, L.C., Florencio, D., Goldenstein, S.: Interpolation of head-related transfer functions using manifold learning. IEEE Signal Process. Lett. 24, 221–225 (2017). https://doi.org/10.1109/LSP.2017.2648794

    Article  Google Scholar 

  44. Hartung, K., Braasch, J., Sterbing, S.J.: Comparison of different methods for the interpolation of head-related transfer functions. In: AES 16th International Conference: Spatial Sound Reproduction, paper 16–028, Audio Engineering Society, Rovaniemi, Finland (1999)

  45. Keyrouz, F., Diepold, K.: A new HRTF interpolation approach for fast synthesis of dynamic environmental interaction. J. Audio Eng. Soc. 56, 28–35 (2008)

    Google Scholar 

  46. Lindau, A., Maempel, H., Weinzierl, S.: Minimum BRIR grid resolution for dynamic binaural synthesis. J. Acoust. Soc. Am. 123, 3498 (2008). https://doi.org/10.1121/1.2934364

    Article  Google Scholar 

  47. Algazi, V.R., Duda, R.O., Thompson, D.M., Avendano, C.: The CIPIC HRTF database. In: 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics, pp. 99–102. IEEE, New Platz, NY, USA (2001)

  48. Huopaniemi, J., Savioja, L., Karjalainen, M.: Modeling of reflections and air absorption in acoustical spaces a digital filter design approach. In: Presented at the Applications of Signal Processing to Audio and Acoustics, 1997. 1997 IEEE ASSP Workshop on (1997)

  49. Ramos, O.A., Araneda, M., Tommasini, F.C.: Diseño y evaluación de filtros binaurales. Mecánica Comput. XXVIII, 137–148 (2009)

    Google Scholar 

  50. Vorländer, M.: Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality. Springer, Berlin (2007)

    Google Scholar 

  51. Beranek, L.L.: Acoustics. American Institute of Physics, New York (1986)

    Google Scholar 

  52. Tommasini, F.C.: Sistema de simulación acústica virtual en tiempo real. Doctoral thesis, Universidad Nacional de Córdoba, Argentina (2012)

  53. Lindau, A.: The perception of system latency in dynamic binaural synthesis. In: Proceedings of NAG/DAGA, pp. 1063–1066. Rotterdam, The Netherlands (2009)

  54. Mackensen, P.: Auditive Localization. Head movements, an additional cue in Localization. Doctoral Thesis, TU Berlin, Germany (2004)

  55. Yairi, S., Iwaya, Y., Suzuki, Y.: Investigation of system latency detection threshold of virtual auditory display. In: Proceedings of ICAD 2006-12th Meeting of the International Conference on Auditory Display, pp. 217–222. London, UK (2006)

  56. Koutsouris, G.I., Brunskog, J., Jeong, C.-H., Jacobsen, F.: Combination of acoustical radiosity and the image source method. J. Acoust. Soc. Am. 133, 3963–3974 (2013). https://doi.org/10.1121/1.4802897

    Article  Google Scholar 

  57. Martellotta, F.: The just noticeable difference of center time and clarity index in large reverberant spaces. J. Acoust. Soc. Am. 128, 654–663 (2010). https://doi.org/10.1121/1.3455837

    Article  Google Scholar 

  58. Hacıhabiboğlu, H., Murtagh, F.: Perceptual simplification for model-based binaural room auralisation. Appl. Acoust. 69, 715–727 (2008). https://doi.org/10.1016/j.apacoust.2007.02.006

    Article  Google Scholar 

  59. Møller, H., Hammershøi, D., Jensen, C.B., Sørensen, M.F.: Transfer characteristics of headphones measured on human ears. J. Audio Eng. Soc. 43, 203–217 (1995)

    Google Scholar 

  60. Bang & Olufsen: Music for Archimedes Audio CD (1992)

  61. Møller, H., Hammershøi, D., Johnson, C.B., Sørensen, M.F.: Evaluation of artificial heads in listening tests. J. Audio Eng. Soc. 47, 83–100 (1999)

    Google Scholar 

  62. Moore, B.C.J., Glasberg, B.R.: Modeling binaural loudness. J. Acoust. Soc. Am. 121, 1604–1612 (2007). https://doi.org/10.1121/1.2431331

    Article  Google Scholar 

  63. Cabrera, D., Ferguson, S., Schubert, E.: PsySound3: An integrated environment for the analysis of sound recordings. In: Acoustics 2008: Proceedings of the Australian Acoustical Society conference (2008)

  64. Wenzel, E.M.: The impact of system latency on dynamic performance in virtual acoustic environments. In: Proceedings of the 16th I International Congress of Acoustics and 135th Meeting of the Acoustical Society of America, p. 180. Seattle, WA (1998)

  65. Middlebrooks, J.C.: Virtual localization improved by scaling nonindividualized external-ear transfer functions in frequency. J. Acoust. Soc. Am. 106, 1493–1510 (1999). https://doi.org/10.1121/1.427147

    Article  Google Scholar 

  66. Shtrepi, L., Astolfi, A., D’Antonio, G., Guski, M.: Objective and perceptual evaluation of distance-dependent scattered sound effects in a small variable-acoustics hall. J. Acoust. Soc. Am. 140, 3651–3662 (2016). https://doi.org/10.1121/1.4966267

    Article  Google Scholar 

  67. Shtrepi, L., Astolfi, A., Puglisi, G.E., Masoero, M.C.: Effects of the distance from a diffusive surface on the objective and perceptual evaluation of the sound field in a small simulated variable-acoustics hall. Appl. Sci. 7, 224 (2017). https://doi.org/10.3390/app7030224

    Article  Google Scholar 

  68. Hodgson, M., York, N., Yang, W., Bliss, M.: Comparison of predicted, measured and auralized sound fields with respect to speech intelligibility in classrooms using CATT-Acoustic and ODEON. Acta Acust. United Acust. 94, 883–890 (2008). https://doi.org/10.3813/AAA.918106

    Article  Google Scholar 

  69. Peng, J.: Feasibility of subjective speech intelligibility assessment based on auralization. Appl. Acoust. 66, 591–601 (2005). https://doi.org/10.1016/j.apacoust.2004.08.006

    Article  Google Scholar 

  70. Peng, J., Bei, C., Sun, H.: Relationship between Chinese speech intelligibility and speech transmission index in rooms based on auralization. Speech Commun. 53, 986–990 (2011). https://doi.org/10.1016/j.specom.2011.05.004

    Article  Google Scholar 

  71. Yang, W., Hodgson, M.: Auralization study of optimum reverberation times for speech intelligibility for normal and hearing-impaired listeners in classrooms with diffuse sound fields. J. Acoust. Soc. Am. 120, 801–807 (2006). https://doi.org/10.1121/1.2216768

    Article  Google Scholar 

  72. Rindel, J.H., Christensen, C.L.: Room acoustic simulation and auralization—how close can we get to the real room? In: Proceedings 8th Western Pacific Acoustics Conference., Melbourne, Australia (2003)

Download references

Acknowledgements

The authors would like to thank María Hinalaf, Ana Luz Maggi, Cecilia Ordoñez, and Karen Grill for their time and contributions to this work and the rest of CINTRA members who gave their support. This work was supported by the Universidad Tecnológica Nacional, Argentina [Grant Numbers PID UTN 982, PID UTN 1705, PID UTN 4498] and the Agencia Nacional de Promoción Científica y Tecnológica, Argentina [Grant Number PICT 2016-0738].

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fabián C. Tommasini.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tommasini, F.C., Ramos, O.A., Hüg, M.X. et al. A Computational Model to Implement Binaural Synthesis in a Hard Real-Time Auditory Virtual Environment. Acoust Aust 47, 51–66 (2019). https://doi.org/10.1007/s40857-019-00152-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40857-019-00152-7

Keywords

Navigation