Abstract
There is a growing interest in the development and the evaluation of real-time auditory virtual environments (AVE). The implementation of this type of simulation system in general purpose computers is a still a challenge, and there are few studies that evaluated the perceived quality of synthetized sounds of simulated acoustic scenes. To evoke in the listener a correct image of the modeling space, the system must be dynamic and interactive. That is, it must respond to the changes in the acoustic scenario produced by the listener movement, in a perceptually acceptable time and with an update rate that guarantees continuity in the reproduction of sound events. Hard real-time systems ensure that a given task runs within a given time interval, providing deterministic behavior for applications with time restrictions. In the current article, a computational model to implement binaural synthesis in a hard real-time AVE is presented and evaluated. The computer model was implemented in an open-source auralization system. Measurements and real-time simulations on a university classroom were carried out to perform a reverberation time parameters validation and a system performance evaluation. Also, measured and simulated binaural soundtracks (composed from anechoic stimuli) were compared in terms of three selected perceptual attributes for subjective evaluations of static positions. The results showed that real-time performance was acceptable according to values previously reported in the literature and that computer prediction errors for the measured parameters were within the subjective difference limens. The computational model was able to generate an AVE with an acceptable overall perceptual quality.
Similar content being viewed by others
References
Kleiner, M., Dalenbäck, B.-I., Svensson, U.P.: Auralization—an overview. J. Audio Eng. Soc. 41, 861–875 (1993)
Savioja, L., Huopaniemi, J., Lokki, T., Väänänen, R.: Creating interactive virtual acoustic environments. J. Audio Eng. Soc. 47, 675–705 (1999)
Lokki, T., Pätynen, J., Tervo, S., Siltanen, S., Savioja, L.: Engaging concert hall acoustics is made up of temporal envelope preserving reflections. J. Acoust. Soc. Am. 129, EL223–EL228 (2011). https://doi.org/10.1121/1.3579145
Bilbao, S., Hamilton, B.: Wave-based room acoustics simulation: explicit/implicit finite volume modeling of viscothermal losses and frequency-dependent boundaries. J. Audio Eng. Soc. 65, 78–89 (2017)
Lentz, T., Schröder, D., Vorländer, M., Assenmacher, I.: Virtual reality system with integrated sound field simulation and reproduction. EURASIP J. Adv. Signal Process. 2007, 1–17 (2007)
Yuan, Y., Fu, Z., Xu, M., Xie, L., Cong, Q.: Externalization improvement in a real-time binaural sound image rendering system. In: 2015 International Conference on Orange Technologies (ICOT), pp. 165–168. IEEE, Hong Kong, China (2015)
Wenzel, E.M., Miller, J.D., Abel, J.S.: Sound Lab: A real-time, software-based system for the study of spatial hearing. In: AES 108th Convention, preprint 5140, Audio Engineering Society, Paris, France (2000)
Scarpaci, J.W.: Creation of a system for real time virtual auditory space and its application to dynamic sound localization. Doctoral thesis, Boston University, Boston, MA (2006)
Noisternig, M., Katz, B.F.G., Siltanen, S., Savioja, L.: Framework for real-time auralization in architectural acoustics. Acta Acust. United Acust. 94, 1000–1015 (2008). https://doi.org/10.3813/AAA.918116
Geier, M., Ahrens, J., Spors, S.: The SoundScape Renderer: A unified spatial audio reproduction framework for arbitrary rendering methods. In: AES 124th Convention, paper 7330, Audio Engineering Society, Amsterdam, The Netherlands (2008)
Blauert, J.: Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge (1997)
Vorländer, M.: Computer simulations in room acoustics: concepts and uncertainties. J. Acoust. Soc. Am. 133, 1203–1213 (2013). https://doi.org/10.1121/1.4788978
Lindau, A., Weinzierl, S.: Assessing the plausibility of virtual acoustic environments. Acta Acust. United Acust. 98, 804–810 (2012). https://doi.org/10.3813/AAA.918562
Pike, C., Melchior, F., Tew, T.: Assessing the plausibility of non-individualised dynamic binaural synthesis in a small room. In: AES 55th International Conference: Spatial Audio, paper 6–1, Audio Engineering Society, Helsinki, Finland (2014)
Brinkmann, F., Lindau, A., Weinzierl, S.: On the authenticity of individual dynamic binaural synthesis. J. Acoust. Soc. Am. 142, 1784–1795 (2017). https://doi.org/10.1121/1.5005606
Langendijk, E.H.A., Bronkhorst, A.W.: Fidelity of three-dimensional-sound reproduction using a virtual auditory display. J. Acoust. Soc. Am. 107, 528–537 (2000). https://doi.org/10.1121/1.428321
Moore, A.H., Tew, A.I., Nicol, R.: An initial validation of individualized crosstalk cancellation filters for binaural perceptual experiments. J. Audio Eng. Soc. 58, 36–45 (2010)
Oberem, J., Masiero, B., Fels, J.: Experiments on authenticity and plausibility of binaural reproduction via headphones employing different recording methods. Appl. Acoust. 114, 71–78 (2016). https://doi.org/10.1016/j.apacoust.2016.07.009
Lindau, A., Erbes, V., Lepa, S., Maempel, H.-J., Brinkman, F., Weinzierl, S.: A spatial audio quality inventory (SAQI). Acta Acust. United Acust. 100, 984–994 (2014). https://doi.org/10.3813/AAA.918778
Lokki, T., Jarvelainen, H.: Subjective evaluation of auralization of physics-based room acoustics modeling. In: Proceedings of the 2001 International Conference on Auditory Display, Espoo, Finland (2001)
Choi, Y.-J., Fricke, F.R.: A comparison of subjective assessments of recorded music and computer simulated auralizations in two auditoria. Acta Acust. United Acust. 92, 604–611 (2006)
Yang, W., Hodgson, M.: Validation of the auralization technique: comparative speech-intelligibility tests in real and virtual classrooms. Acta Acust. United Acust. 93, 991–999 (2007)
Postma, B.N.J., Katz, B.F.G.: Perceptive and objective evaluation of calibrated room acoustic simulation auralizations. J. Acoust. Soc. Am. 140, 4326–4337 (2016). https://doi.org/10.1121/1.4971422
Tommasini, F.C., Ramos, O.A., Ferreyra, S., Guido, R.M.: Sistema de realidad acústica virtual en tiempo real: AVRS. In: Proceedings of IX Congreso Iberoamericano de Acústica (FIA 2014), Valdivia, Chile (2014)
Mantegazza, P., Dozio, E.L., Papacharalambous, S.: RTAI: real time application interface. Linux J. 2000, 10 (2000)
Arm, J., Bradac, Z., Kaczmarczyk, V.: Real-time capabilities of Linux RTAI. IFAC Pap. 49, 401–406 (2016). https://doi.org/10.1016/j.ifacol.2016.12.080
Allen, J.B., Berkley, D.A.: Image method for efficiently simulating small-room acoustics. J. Acoust. Soc. Am. 65, 943–950 (1979)
Borish, J.: Extension of the image model to arbitrary polyhedra. J. Acoust. Soc. Am. 75, 1827–1836 (1984). https://doi.org/10.1121/1.390983
Bradley, J.S., Sato, H., Picard, M.: On the importance of early reflections for speech in rooms. J. Acoust. Soc. Am. 113, 3233–3244 (2003). https://doi.org/10.1121/1.1570439
Jot, J.-M.: Efficient models for reverberation and distance rendering in computer music and virtual audio reality. In: International Computer Music Conference Proceedings 1997 (1997)
Schlecht, S.J., Habets, E.A.P.: Feedback delay networks: echo density and mixing time. IEEEACM Trans. Audio Speech Lang. Process. 25, 374–383 (2017). https://doi.org/10.1109/TASLP.2016.2635027
Wendt, T., van de Par, S., Ewert, S.D.: A computationally-efficient and perceptually-plausible algorithm for binaural room impulse response simulation. J. Audio Eng. Soc. 62, 748–766 (2014)
Dalenbäck, B.-I.L.: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. J. Acoust. Soc. Am. 100, 899–909 (1996). https://doi.org/10.1121/1.416249
Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J.E., Pingali, G., Min, P., Ngan, A.: A beam tracing method for interactive architectural acoustics. J. Acoust. Soc. Am. 115, 739 (2004). https://doi.org/10.1121/1.1641020
Vorländer, M.: Simulation of the transient and steady-state sound propagation in rooms using a new combined ray-tracing/image-source algorithm. J. Acoust. Soc. Am. 86, 172–178 (1989). https://doi.org/10.1121/1.398336
Lehmann, E.A., Johansson, A.M.: Diffuse reverberation model for efficient image-source simulation of room impulse responses. IEEE Trans. Audio Speech Lang. Process. 18, 1429–1439 (2010). https://doi.org/10.1109/TASL.2009.2035038
Lehmann, E.A., Johansson, A.M.: Prediction of energy decay in room impulse responses simulated with an image-source model. J. Acoust. Soc. Am. 124, 269–277 (2008). https://doi.org/10.1121/1.2936367
Cremer, L., Müller, H.A.: Principles and applications of room acoustics. Appl. Sci. 1, (1982)
Defrance, G., Polack, J.: Measuring the mixing time in auditoria. J. Acoust. Soc. Am. 123, 3499 (2008). https://doi.org/10.1121/1.2934368
Hidaka, T., Yamada, Y., Nakagawa, T.: A new definition of boundary point between early reflections and late reverberation in room impulse responses. J. Acoust. Soc. Am. 122, 326–332 (2007). https://doi.org/10.1121/1.2743161
Lindau, A., Kosanke, L., Weinzierl, S.: Perceptual evaluation of physical predictors of the mixing time in binaural room impulse responses. In: AES 128th Convention, paper 8089, Audio Engineering Society, London, UK (2010)
Defrance, G., Polack, J.: Estimating the mixing time of concert halls using the eXtensible Fourier Transform. Appl. Acoust. 71, 777–792 (2010). https://doi.org/10.1016/j.apacoust.2010.05.011
Grijalva, F., Martini, L.C., Florencio, D., Goldenstein, S.: Interpolation of head-related transfer functions using manifold learning. IEEE Signal Process. Lett. 24, 221–225 (2017). https://doi.org/10.1109/LSP.2017.2648794
Hartung, K., Braasch, J., Sterbing, S.J.: Comparison of different methods for the interpolation of head-related transfer functions. In: AES 16th International Conference: Spatial Sound Reproduction, paper 16–028, Audio Engineering Society, Rovaniemi, Finland (1999)
Keyrouz, F., Diepold, K.: A new HRTF interpolation approach for fast synthesis of dynamic environmental interaction. J. Audio Eng. Soc. 56, 28–35 (2008)
Lindau, A., Maempel, H., Weinzierl, S.: Minimum BRIR grid resolution for dynamic binaural synthesis. J. Acoust. Soc. Am. 123, 3498 (2008). https://doi.org/10.1121/1.2934364
Algazi, V.R., Duda, R.O., Thompson, D.M., Avendano, C.: The CIPIC HRTF database. In: 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics, pp. 99–102. IEEE, New Platz, NY, USA (2001)
Huopaniemi, J., Savioja, L., Karjalainen, M.: Modeling of reflections and air absorption in acoustical spaces a digital filter design approach. In: Presented at the Applications of Signal Processing to Audio and Acoustics, 1997. 1997 IEEE ASSP Workshop on (1997)
Ramos, O.A., Araneda, M., Tommasini, F.C.: Diseño y evaluación de filtros binaurales. Mecánica Comput. XXVIII, 137–148 (2009)
Vorländer, M.: Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality. Springer, Berlin (2007)
Beranek, L.L.: Acoustics. American Institute of Physics, New York (1986)
Tommasini, F.C.: Sistema de simulación acústica virtual en tiempo real. Doctoral thesis, Universidad Nacional de Córdoba, Argentina (2012)
Lindau, A.: The perception of system latency in dynamic binaural synthesis. In: Proceedings of NAG/DAGA, pp. 1063–1066. Rotterdam, The Netherlands (2009)
Mackensen, P.: Auditive Localization. Head movements, an additional cue in Localization. Doctoral Thesis, TU Berlin, Germany (2004)
Yairi, S., Iwaya, Y., Suzuki, Y.: Investigation of system latency detection threshold of virtual auditory display. In: Proceedings of ICAD 2006-12th Meeting of the International Conference on Auditory Display, pp. 217–222. London, UK (2006)
Koutsouris, G.I., Brunskog, J., Jeong, C.-H., Jacobsen, F.: Combination of acoustical radiosity and the image source method. J. Acoust. Soc. Am. 133, 3963–3974 (2013). https://doi.org/10.1121/1.4802897
Martellotta, F.: The just noticeable difference of center time and clarity index in large reverberant spaces. J. Acoust. Soc. Am. 128, 654–663 (2010). https://doi.org/10.1121/1.3455837
Hacıhabiboğlu, H., Murtagh, F.: Perceptual simplification for model-based binaural room auralisation. Appl. Acoust. 69, 715–727 (2008). https://doi.org/10.1016/j.apacoust.2007.02.006
Møller, H., Hammershøi, D., Jensen, C.B., Sørensen, M.F.: Transfer characteristics of headphones measured on human ears. J. Audio Eng. Soc. 43, 203–217 (1995)
Bang & Olufsen: Music for Archimedes Audio CD (1992)
Møller, H., Hammershøi, D., Johnson, C.B., Sørensen, M.F.: Evaluation of artificial heads in listening tests. J. Audio Eng. Soc. 47, 83–100 (1999)
Moore, B.C.J., Glasberg, B.R.: Modeling binaural loudness. J. Acoust. Soc. Am. 121, 1604–1612 (2007). https://doi.org/10.1121/1.2431331
Cabrera, D., Ferguson, S., Schubert, E.: PsySound3: An integrated environment for the analysis of sound recordings. In: Acoustics 2008: Proceedings of the Australian Acoustical Society conference (2008)
Wenzel, E.M.: The impact of system latency on dynamic performance in virtual acoustic environments. In: Proceedings of the 16th I International Congress of Acoustics and 135th Meeting of the Acoustical Society of America, p. 180. Seattle, WA (1998)
Middlebrooks, J.C.: Virtual localization improved by scaling nonindividualized external-ear transfer functions in frequency. J. Acoust. Soc. Am. 106, 1493–1510 (1999). https://doi.org/10.1121/1.427147
Shtrepi, L., Astolfi, A., D’Antonio, G., Guski, M.: Objective and perceptual evaluation of distance-dependent scattered sound effects in a small variable-acoustics hall. J. Acoust. Soc. Am. 140, 3651–3662 (2016). https://doi.org/10.1121/1.4966267
Shtrepi, L., Astolfi, A., Puglisi, G.E., Masoero, M.C.: Effects of the distance from a diffusive surface on the objective and perceptual evaluation of the sound field in a small simulated variable-acoustics hall. Appl. Sci. 7, 224 (2017). https://doi.org/10.3390/app7030224
Hodgson, M., York, N., Yang, W., Bliss, M.: Comparison of predicted, measured and auralized sound fields with respect to speech intelligibility in classrooms using CATT-Acoustic and ODEON. Acta Acust. United Acust. 94, 883–890 (2008). https://doi.org/10.3813/AAA.918106
Peng, J.: Feasibility of subjective speech intelligibility assessment based on auralization. Appl. Acoust. 66, 591–601 (2005). https://doi.org/10.1016/j.apacoust.2004.08.006
Peng, J., Bei, C., Sun, H.: Relationship between Chinese speech intelligibility and speech transmission index in rooms based on auralization. Speech Commun. 53, 986–990 (2011). https://doi.org/10.1016/j.specom.2011.05.004
Yang, W., Hodgson, M.: Auralization study of optimum reverberation times for speech intelligibility for normal and hearing-impaired listeners in classrooms with diffuse sound fields. J. Acoust. Soc. Am. 120, 801–807 (2006). https://doi.org/10.1121/1.2216768
Rindel, J.H., Christensen, C.L.: Room acoustic simulation and auralization—how close can we get to the real room? In: Proceedings 8th Western Pacific Acoustics Conference., Melbourne, Australia (2003)
Acknowledgements
The authors would like to thank María Hinalaf, Ana Luz Maggi, Cecilia Ordoñez, and Karen Grill for their time and contributions to this work and the rest of CINTRA members who gave their support. This work was supported by the Universidad Tecnológica Nacional, Argentina [Grant Numbers PID UTN 982, PID UTN 1705, PID UTN 4498] and the Agencia Nacional de Promoción Científica y Tecnológica, Argentina [Grant Number PICT 2016-0738].
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tommasini, F.C., Ramos, O.A., Hüg, M.X. et al. A Computational Model to Implement Binaural Synthesis in a Hard Real-Time Auditory Virtual Environment. Acoust Aust 47, 51–66 (2019). https://doi.org/10.1007/s40857-019-00152-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40857-019-00152-7