Sound Source Localization Using Beamforming and Its Representation in a Mixed Reality Embedded Device
Acoustic localization is a technique that allows measuring the intensity and direction of a sound source. A popular technique for its calculation is the beamforming algorithm “Sum-and-Delay” which generates an approximate angle of its location while reducing noise in the signal. Combining this technique with advances in mixed reality environments it is possible to develop essential applications both in the industry and beyond. However, these systems can be too large and expensive. In this work, a method was designed to approximate the sound direction of arrival using a beamforming technique to represent it in mixed reality environment using an embedded device, achieving to approximate it with an average error of 2.5% in distances up to 3 m long. Among its main advantages is that it is a portable and low-cost system in which applications of great utility can be developed, such as mixed reality conversation assistants capable of generating subtitles in real-time to assist people with deafness in a classrooms that do not have a translator, or to incorporate a translation service to understand speakers in different languages.
KeywordsSound source localization Direction of arrival Mixed reality Beamforming Embedded hardware
The authors wish to acknowledge the financial support for this work by the Consejo Nacional de Ciencia y Tecnología (CONACYT) through financial support scholarship number 494038. We also want to thank Universidad Autónoma de Querétaro (UAQ) through project number FIF-2018-06.
- 4.Huang, Z., Hui, P., Peylo, C., Chatzopoulos, D.: Mobile augmented reality survey: a bottom-up approach. CoRR, abs/1309.4413, p. 1 (2013)Google Scholar
- 6.Wang, H., Chu, P.: Voice source localization for automatic camera pointing system in videoconferencing. In: 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (1997). https://doi.org/10.1109/icassp.1997.599595
- 8.Huang, Y., Benesty, J., Elko, G.: Passive acoustic source localization for video camera steering. In: 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100) (2000). https://doi.org/10.1109/icassp.2000.859108
- 9.Lanslots, J., Deblauwe, F., Janssens, K.: Selecting sound source localization techniques for industrial applications. Sound Vib. (2010). http://www.sandv.com/home.htm. Accessed 15 Feb 2019
- 10.Azar, J., Abou Saleh, H., Al-Alaoui, M.: Sound visualization for the hearing impaired. Int. J. Emerg. Technol. Learn. (iJET) 2(1) (2007). International Association of Online Engineering, Kassel, Germany. https://www.learntechlib.org/p/45106/. Accessed 15 Feb 2019
- 11.Jain, D., et al.: Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015 (2015)Google Scholar
- 12.Greensted, A.: Delay Sum Beamforming - The Lab Book Pages (2012). http://www.labbookpages.co.uk/audio/beamforming/delaySum.html
- 13.Pessentheiner, H.: Beamforming Using Uniform Circular Arrays for Distant Speech Recognition in Reverberant Environment and Double-Talk Scenarios. Bachelor of Science, Graz University of Technology (2012)Google Scholar
- 14.Acero, A., Stern, R.: Environmental robustness in automatic speech recognition. In: International Conference on Acoustics, Speech, and Signal Processing. https://doi.org/10.1109/icassp.1990.115971
- 15.Grondin, F., Michaud, F.: Lightweight and optimized sound source localization and tracking methods for opened and closed microphone array configurations. Rob. Auton. Syst. (2019). https://doi.org/10.1016/j.robot.2019.01.002