Advertisement

Acoustics Australia

, Volume 47, Issue 1, pp 51–66 | Cite as

A Computational Model to Implement Binaural Synthesis in a Hard Real-Time Auditory Virtual Environment

  • Fabián C. TommasiniEmail author
  • Oscar A. Ramos
  • Mercedes X. Hüg
  • Sebastián P. Ferreyra
Original Paper
  • 110 Downloads

Abstract

There is a growing interest in the development and the evaluation of real-time auditory virtual environments (AVE). The implementation of this type of simulation system in general purpose computers is a still a challenge, and there are few studies that evaluated the perceived quality of synthetized sounds of simulated acoustic scenes. To evoke in the listener a correct image of the modeling space, the system must be dynamic and interactive. That is, it must respond to the changes in the acoustic scenario produced by the listener movement, in a perceptually acceptable time and with an update rate that guarantees continuity in the reproduction of sound events. Hard real-time systems ensure that a given task runs within a given time interval, providing deterministic behavior for applications with time restrictions. In the current article, a computational model to implement binaural synthesis in a hard real-time AVE is presented and evaluated. The computer model was implemented in an open-source auralization system. Measurements and real-time simulations on a university classroom were carried out to perform a reverberation time parameters validation and a system performance evaluation. Also, measured and simulated binaural soundtracks (composed from anechoic stimuli) were compared in terms of three selected perceptual attributes for subjective evaluations of static positions. The results showed that real-time performance was acceptable according to values previously reported in the literature and that computer prediction errors for the measured parameters were within the subjective difference limens. The computational model was able to generate an AVE with an acceptable overall perceptual quality.

Keywords

Auditory virtual environment Real-time auralization Binaural synthesis performance Perceptual quality evaluation 

Notes

Acknowledgements

The authors would like to thank María Hinalaf, Ana Luz Maggi, Cecilia Ordoñez, and Karen Grill for their time and contributions to this work and the rest of CINTRA members who gave their support. This work was supported by the Universidad Tecnológica Nacional, Argentina [Grant Numbers PID UTN 982, PID UTN 1705, PID UTN 4498] and the Agencia Nacional de Promoción Científica y Tecnológica, Argentina [Grant Number PICT 2016-0738].

References

  1. 1.
    Kleiner, M., Dalenbäck, B.-I., Svensson, U.P.: Auralization—an overview. J. Audio Eng. Soc. 41, 861–875 (1993)Google Scholar
  2. 2.
    Savioja, L., Huopaniemi, J., Lokki, T., Väänänen, R.: Creating interactive virtual acoustic environments. J. Audio Eng. Soc. 47, 675–705 (1999)Google Scholar
  3. 3.
    Lokki, T., Pätynen, J., Tervo, S., Siltanen, S., Savioja, L.: Engaging concert hall acoustics is made up of temporal envelope preserving reflections. J. Acoust. Soc. Am. 129, EL223–EL228 (2011).  https://doi.org/10.1121/1.3579145 CrossRefGoogle Scholar
  4. 4.
    Bilbao, S., Hamilton, B.: Wave-based room acoustics simulation: explicit/implicit finite volume modeling of viscothermal losses and frequency-dependent boundaries. J. Audio Eng. Soc. 65, 78–89 (2017)CrossRefGoogle Scholar
  5. 5.
    Lentz, T., Schröder, D., Vorländer, M., Assenmacher, I.: Virtual reality system with integrated sound field simulation and reproduction. EURASIP J. Adv. Signal Process. 2007, 1–17 (2007)CrossRefGoogle Scholar
  6. 6.
    Yuan, Y., Fu, Z., Xu, M., Xie, L., Cong, Q.: Externalization improvement in a real-time binaural sound image rendering system. In: 2015 International Conference on Orange Technologies (ICOT), pp. 165–168. IEEE, Hong Kong, China (2015)Google Scholar
  7. 7.
    Wenzel, E.M., Miller, J.D., Abel, J.S.: Sound Lab: A real-time, software-based system for the study of spatial hearing. In: AES 108th Convention, preprint 5140, Audio Engineering Society, Paris, France (2000)Google Scholar
  8. 8.
    Scarpaci, J.W.: Creation of a system for real time virtual auditory space and its application to dynamic sound localization. Doctoral thesis, Boston University, Boston, MA (2006)Google Scholar
  9. 9.
    Noisternig, M., Katz, B.F.G., Siltanen, S., Savioja, L.: Framework for real-time auralization in architectural acoustics. Acta Acust. United Acust. 94, 1000–1015 (2008).  https://doi.org/10.3813/AAA.918116 CrossRefGoogle Scholar
  10. 10.
    Geier, M., Ahrens, J., Spors, S.: The SoundScape Renderer: A unified spatial audio reproduction framework for arbitrary rendering methods. In: AES 124th Convention, paper 7330, Audio Engineering Society, Amsterdam, The Netherlands (2008)Google Scholar
  11. 11.
    Blauert, J.: Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge (1997)Google Scholar
  12. 12.
    Vorländer, M.: Computer simulations in room acoustics: concepts and uncertainties. J. Acoust. Soc. Am. 133, 1203–1213 (2013).  https://doi.org/10.1121/1.4788978 CrossRefGoogle Scholar
  13. 13.
    Lindau, A., Weinzierl, S.: Assessing the plausibility of virtual acoustic environments. Acta Acust. United Acust. 98, 804–810 (2012).  https://doi.org/10.3813/AAA.918562 CrossRefGoogle Scholar
  14. 14.
    Pike, C., Melchior, F., Tew, T.: Assessing the plausibility of non-individualised dynamic binaural synthesis in a small room. In: AES 55th International Conference: Spatial Audio, paper 6–1, Audio Engineering Society, Helsinki, Finland (2014)Google Scholar
  15. 15.
    Brinkmann, F., Lindau, A., Weinzierl, S.: On the authenticity of individual dynamic binaural synthesis. J. Acoust. Soc. Am. 142, 1784–1795 (2017).  https://doi.org/10.1121/1.5005606 CrossRefGoogle Scholar
  16. 16.
    Langendijk, E.H.A., Bronkhorst, A.W.: Fidelity of three-dimensional-sound reproduction using a virtual auditory display. J. Acoust. Soc. Am. 107, 528–537 (2000).  https://doi.org/10.1121/1.428321 CrossRefGoogle Scholar
  17. 17.
    Moore, A.H., Tew, A.I., Nicol, R.: An initial validation of individualized crosstalk cancellation filters for binaural perceptual experiments. J. Audio Eng. Soc. 58, 36–45 (2010)Google Scholar
  18. 18.
    Oberem, J., Masiero, B., Fels, J.: Experiments on authenticity and plausibility of binaural reproduction via headphones employing different recording methods. Appl. Acoust. 114, 71–78 (2016).  https://doi.org/10.1016/j.apacoust.2016.07.009 CrossRefGoogle Scholar
  19. 19.
    Lindau, A., Erbes, V., Lepa, S., Maempel, H.-J., Brinkman, F., Weinzierl, S.: A spatial audio quality inventory (SAQI). Acta Acust. United Acust. 100, 984–994 (2014).  https://doi.org/10.3813/AAA.918778 CrossRefGoogle Scholar
  20. 20.
    Lokki, T., Jarvelainen, H.: Subjective evaluation of auralization of physics-based room acoustics modeling. In: Proceedings of the 2001 International Conference on Auditory Display, Espoo, Finland (2001)Google Scholar
  21. 21.
    Choi, Y.-J., Fricke, F.R.: A comparison of subjective assessments of recorded music and computer simulated auralizations in two auditoria. Acta Acust. United Acust. 92, 604–611 (2006)Google Scholar
  22. 22.
    Yang, W., Hodgson, M.: Validation of the auralization technique: comparative speech-intelligibility tests in real and virtual classrooms. Acta Acust. United Acust. 93, 991–999 (2007)Google Scholar
  23. 23.
    Postma, B.N.J., Katz, B.F.G.: Perceptive and objective evaluation of calibrated room acoustic simulation auralizations. J. Acoust. Soc. Am. 140, 4326–4337 (2016).  https://doi.org/10.1121/1.4971422 CrossRefGoogle Scholar
  24. 24.
    Tommasini, F.C., Ramos, O.A., Ferreyra, S., Guido, R.M.: Sistema de realidad acústica virtual en tiempo real: AVRS. In: Proceedings of IX Congreso Iberoamericano de Acústica (FIA 2014), Valdivia, Chile (2014)Google Scholar
  25. 25.
    Mantegazza, P., Dozio, E.L., Papacharalambous, S.: RTAI: real time application interface. Linux J. 2000, 10 (2000)Google Scholar
  26. 26.
    Arm, J., Bradac, Z., Kaczmarczyk, V.: Real-time capabilities of Linux RTAI. IFAC Pap. 49, 401–406 (2016).  https://doi.org/10.1016/j.ifacol.2016.12.080 CrossRefGoogle Scholar
  27. 27.
    Allen, J.B., Berkley, D.A.: Image method for efficiently simulating small-room acoustics. J. Acoust. Soc. Am. 65, 943–950 (1979)CrossRefGoogle Scholar
  28. 28.
    Borish, J.: Extension of the image model to arbitrary polyhedra. J. Acoust. Soc. Am. 75, 1827–1836 (1984).  https://doi.org/10.1121/1.390983 CrossRefGoogle Scholar
  29. 29.
    Bradley, J.S., Sato, H., Picard, M.: On the importance of early reflections for speech in rooms. J. Acoust. Soc. Am. 113, 3233–3244 (2003).  https://doi.org/10.1121/1.1570439 CrossRefGoogle Scholar
  30. 30.
    Jot, J.-M.: Efficient models for reverberation and distance rendering in computer music and virtual audio reality. In: International Computer Music Conference Proceedings 1997 (1997)Google Scholar
  31. 31.
    Schlecht, S.J., Habets, E.A.P.: Feedback delay networks: echo density and mixing time. IEEEACM Trans. Audio Speech Lang. Process. 25, 374–383 (2017).  https://doi.org/10.1109/TASLP.2016.2635027 CrossRefGoogle Scholar
  32. 32.
    Wendt, T., van de Par, S., Ewert, S.D.: A computationally-efficient and perceptually-plausible algorithm for binaural room impulse response simulation. J. Audio Eng. Soc. 62, 748–766 (2014)CrossRefGoogle Scholar
  33. 33.
    Dalenbäck, B.-I.L.: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. J. Acoust. Soc. Am. 100, 899–909 (1996).  https://doi.org/10.1121/1.416249 CrossRefGoogle Scholar
  34. 34.
    Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J.E., Pingali, G., Min, P., Ngan, A.: A beam tracing method for interactive architectural acoustics. J. Acoust. Soc. Am. 115, 739 (2004).  https://doi.org/10.1121/1.1641020 CrossRefGoogle Scholar
  35. 35.
    Vorländer, M.: Simulation of the transient and steady-state sound propagation in rooms using a new combined ray-tracing/image-source algorithm. J. Acoust. Soc. Am. 86, 172–178 (1989).  https://doi.org/10.1121/1.398336 CrossRefGoogle Scholar
  36. 36.
    Lehmann, E.A., Johansson, A.M.: Diffuse reverberation model for efficient image-source simulation of room impulse responses. IEEE Trans. Audio Speech Lang. Process. 18, 1429–1439 (2010).  https://doi.org/10.1109/TASL.2009.2035038 CrossRefGoogle Scholar
  37. 37.
    Lehmann, E.A., Johansson, A.M.: Prediction of energy decay in room impulse responses simulated with an image-source model. J. Acoust. Soc. Am. 124, 269–277 (2008).  https://doi.org/10.1121/1.2936367 CrossRefGoogle Scholar
  38. 38.
    Cremer, L., Müller, H.A.: Principles and applications of room acoustics. Appl. Sci. 1, (1982)Google Scholar
  39. 39.
    Defrance, G., Polack, J.: Measuring the mixing time in auditoria. J. Acoust. Soc. Am. 123, 3499 (2008).  https://doi.org/10.1121/1.2934368 CrossRefGoogle Scholar
  40. 40.
    Hidaka, T., Yamada, Y., Nakagawa, T.: A new definition of boundary point between early reflections and late reverberation in room impulse responses. J. Acoust. Soc. Am. 122, 326–332 (2007).  https://doi.org/10.1121/1.2743161 CrossRefGoogle Scholar
  41. 41.
    Lindau, A., Kosanke, L., Weinzierl, S.: Perceptual evaluation of physical predictors of the mixing time in binaural room impulse responses. In: AES 128th Convention, paper 8089, Audio Engineering Society, London, UK (2010)Google Scholar
  42. 42.
    Defrance, G., Polack, J.: Estimating the mixing time of concert halls using the eXtensible Fourier Transform. Appl. Acoust. 71, 777–792 (2010).  https://doi.org/10.1016/j.apacoust.2010.05.011 CrossRefGoogle Scholar
  43. 43.
    Grijalva, F., Martini, L.C., Florencio, D., Goldenstein, S.: Interpolation of head-related transfer functions using manifold learning. IEEE Signal Process. Lett. 24, 221–225 (2017).  https://doi.org/10.1109/LSP.2017.2648794 CrossRefGoogle Scholar
  44. 44.
    Hartung, K., Braasch, J., Sterbing, S.J.: Comparison of different methods for the interpolation of head-related transfer functions. In: AES 16th International Conference: Spatial Sound Reproduction, paper 16–028, Audio Engineering Society, Rovaniemi, Finland (1999)Google Scholar
  45. 45.
    Keyrouz, F., Diepold, K.: A new HRTF interpolation approach for fast synthesis of dynamic environmental interaction. J. Audio Eng. Soc. 56, 28–35 (2008)Google Scholar
  46. 46.
    Lindau, A., Maempel, H., Weinzierl, S.: Minimum BRIR grid resolution for dynamic binaural synthesis. J. Acoust. Soc. Am. 123, 3498 (2008).  https://doi.org/10.1121/1.2934364 CrossRefGoogle Scholar
  47. 47.
    Algazi, V.R., Duda, R.O., Thompson, D.M., Avendano, C.: The CIPIC HRTF database. In: 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics, pp. 99–102. IEEE, New Platz, NY, USA (2001)Google Scholar
  48. 48.
    Huopaniemi, J., Savioja, L., Karjalainen, M.: Modeling of reflections and air absorption in acoustical spaces a digital filter design approach. In: Presented at the Applications of Signal Processing to Audio and Acoustics, 1997. 1997 IEEE ASSP Workshop on (1997)Google Scholar
  49. 49.
    Ramos, O.A., Araneda, M., Tommasini, F.C.: Diseño y evaluación de filtros binaurales. Mecánica Comput. XXVIII, 137–148 (2009)Google Scholar
  50. 50.
    Vorländer, M.: Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality. Springer, Berlin (2007)Google Scholar
  51. 51.
    Beranek, L.L.: Acoustics. American Institute of Physics, New York (1986)Google Scholar
  52. 52.
    Tommasini, F.C.: Sistema de simulación acústica virtual en tiempo real. Doctoral thesis, Universidad Nacional de Córdoba, Argentina (2012)Google Scholar
  53. 53.
    Lindau, A.: The perception of system latency in dynamic binaural synthesis. In: Proceedings of NAG/DAGA, pp. 1063–1066. Rotterdam, The Netherlands (2009)Google Scholar
  54. 54.
    Mackensen, P.: Auditive Localization. Head movements, an additional cue in Localization. Doctoral Thesis, TU Berlin, Germany (2004)Google Scholar
  55. 55.
    Yairi, S., Iwaya, Y., Suzuki, Y.: Investigation of system latency detection threshold of virtual auditory display. In: Proceedings of ICAD 2006-12th Meeting of the International Conference on Auditory Display, pp. 217–222. London, UK (2006)Google Scholar
  56. 56.
    Koutsouris, G.I., Brunskog, J., Jeong, C.-H., Jacobsen, F.: Combination of acoustical radiosity and the image source method. J. Acoust. Soc. Am. 133, 3963–3974 (2013).  https://doi.org/10.1121/1.4802897 CrossRefGoogle Scholar
  57. 57.
    Martellotta, F.: The just noticeable difference of center time and clarity index in large reverberant spaces. J. Acoust. Soc. Am. 128, 654–663 (2010).  https://doi.org/10.1121/1.3455837 CrossRefGoogle Scholar
  58. 58.
    Hacıhabiboğlu, H., Murtagh, F.: Perceptual simplification for model-based binaural room auralisation. Appl. Acoust. 69, 715–727 (2008).  https://doi.org/10.1016/j.apacoust.2007.02.006 CrossRefGoogle Scholar
  59. 59.
    Møller, H., Hammershøi, D., Jensen, C.B., Sørensen, M.F.: Transfer characteristics of headphones measured on human ears. J. Audio Eng. Soc. 43, 203–217 (1995)Google Scholar
  60. 60.
    Bang & Olufsen: Music for Archimedes Audio CD (1992)Google Scholar
  61. 61.
    Møller, H., Hammershøi, D., Johnson, C.B., Sørensen, M.F.: Evaluation of artificial heads in listening tests. J. Audio Eng. Soc. 47, 83–100 (1999)Google Scholar
  62. 62.
    Moore, B.C.J., Glasberg, B.R.: Modeling binaural loudness. J. Acoust. Soc. Am. 121, 1604–1612 (2007).  https://doi.org/10.1121/1.2431331 CrossRefGoogle Scholar
  63. 63.
    Cabrera, D., Ferguson, S., Schubert, E.: PsySound3: An integrated environment for the analysis of sound recordings. In: Acoustics 2008: Proceedings of the Australian Acoustical Society conference (2008)Google Scholar
  64. 64.
    Wenzel, E.M.: The impact of system latency on dynamic performance in virtual acoustic environments. In: Proceedings of the 16th I International Congress of Acoustics and 135th Meeting of the Acoustical Society of America, p. 180. Seattle, WA (1998)Google Scholar
  65. 65.
    Middlebrooks, J.C.: Virtual localization improved by scaling nonindividualized external-ear transfer functions in frequency. J. Acoust. Soc. Am. 106, 1493–1510 (1999).  https://doi.org/10.1121/1.427147 CrossRefGoogle Scholar
  66. 66.
    Shtrepi, L., Astolfi, A., D’Antonio, G., Guski, M.: Objective and perceptual evaluation of distance-dependent scattered sound effects in a small variable-acoustics hall. J. Acoust. Soc. Am. 140, 3651–3662 (2016).  https://doi.org/10.1121/1.4966267 CrossRefGoogle Scholar
  67. 67.
    Shtrepi, L., Astolfi, A., Puglisi, G.E., Masoero, M.C.: Effects of the distance from a diffusive surface on the objective and perceptual evaluation of the sound field in a small simulated variable-acoustics hall. Appl. Sci. 7, 224 (2017).  https://doi.org/10.3390/app7030224 CrossRefGoogle Scholar
  68. 68.
    Hodgson, M., York, N., Yang, W., Bliss, M.: Comparison of predicted, measured and auralized sound fields with respect to speech intelligibility in classrooms using CATT-Acoustic and ODEON. Acta Acust. United Acust. 94, 883–890 (2008).  https://doi.org/10.3813/AAA.918106 CrossRefGoogle Scholar
  69. 69.
    Peng, J.: Feasibility of subjective speech intelligibility assessment based on auralization. Appl. Acoust. 66, 591–601 (2005).  https://doi.org/10.1016/j.apacoust.2004.08.006 CrossRefGoogle Scholar
  70. 70.
    Peng, J., Bei, C., Sun, H.: Relationship between Chinese speech intelligibility and speech transmission index in rooms based on auralization. Speech Commun. 53, 986–990 (2011).  https://doi.org/10.1016/j.specom.2011.05.004 CrossRefGoogle Scholar
  71. 71.
    Yang, W., Hodgson, M.: Auralization study of optimum reverberation times for speech intelligibility for normal and hearing-impaired listeners in classrooms with diffuse sound fields. J. Acoust. Soc. Am. 120, 801–807 (2006).  https://doi.org/10.1121/1.2216768 CrossRefGoogle Scholar
  72. 72.
    Rindel, J.H., Christensen, C.L.: Room acoustic simulation and auralization—how close can we get to the real room? In: Proceedings 8th Western Pacific Acoustics Conference., Melbourne, Australia (2003)Google Scholar

Copyright information

© Australian Acoustical Society 2019

Authors and Affiliations

  1. 1.Centro de Investigación y Transferencia en Acústica (CINTRA)Universidad Tecnológica Nacional - Facultad Regional Córdoba, CONICET. Maestro M. López esq. Cruz Roja ArgentinaCórdobaArgentina
  2. 2.Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)Buenos AiresArgentina
  3. 3.Facultad de PsicologíaUniversidad Nacional de Córdoba, Bv. de la Reforma esq. Enfermera GordilloCórdobaArgentina

Personalised recommendations