Abstract
Audio description, an accessibility service used by blind or visually impaired individuals, provides spoken descriptions of visual content. This alternative format allows those with low or no vision the ability to access information that sighted people obtain visually. In this paper a method for deploying prerecorded audio description in a live musical theater environment is presented. This method uses a reference audio recording and an online time warping algorithm to align tracks of audio description with live performances. A software implementation that is integrated into an existing theatrical workflow is also described. This system is used in two evaluation experiments that show the method successfully aligns multiple recordings of works of musical theater in order to automatically trigger prerecorded, descriptive audio in real time.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arzt, A.: Flexible and robust music tracking. Ph.D. dissertation, Johannes Kepler University, Linz (2016)
Arzt, A., Widmer, G., Dixon, S.: Automatic page turning for musicians via real-time machine listening. In: Proceedings of the European Conference on Artificial Intelligence, pp. 241–245 (2008)
Branje, C.J., Fels, D.I.: LiveDescribe: can amateur describers create high-quality audio description? J. Vis. Impair. Blind. 106(3), 154–165 (2012)
Campos, V.P., de Araujo, T.M.U., de Souza Filho, G.L., Goncalves, L.M.G.: CineAD: a system for automated audio description script generation for the visually impaired. Universal Access in the Information Society, pp. 1–13 (2018)
Dixon, S.: Live tracking of musical performances using on-line time warping. In: Proceedings of the 8th International Conference on Digital Audio Effects, pp. 92–97 (2005)
Dubagunta, S.P.: A simple MFCC extractor using C++ STL and C++11. Source code at (2016). http://www.github.com/dspavankumar/compute-mfcc
Fryer, L.: An Introduction to Audio Description: A Practical Guide. Routledge, London (2016)
Lertwongkhanakool, N., Kertkeidkachorn, N., Punyabukkana, P., Suchato, A.: An automatic real-time synchronization of live speech with its transcription approach. Eng. J. 19(5), 81–99 (2015)
Litsyn, E., Pipko, H.: System and method for distribution and synchronized presentation of content. U.S. Patent Application 16/092,775, 2 May 2019
Logan, B.: Mel frequency cepstral coefficients for music modeling. ISMIR. 270, 1–11 (2000)
Muda, L., Begam, M., Elamvazuthi, I.: Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. J. Comput. 2(3) (2010)
Plaza, M.: Cost-effectiveness of audio description process: a comparative analysis of outsourcing and “in-house" methods. Int. J. Prod. Res. 55, 3480–3496 (2017)
Sakoe, H., Chiba, S.: Dynamic programming algorithm optimisation for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26, 43–49 (1978)
Snyder, J.: The visual made verbal: A comprehensive training manual and guide to the history and applications of audio description. American Council of the Blind (2014)
Szarkowska, A.: Text-to-speech audio description: towards a wider availability of AD. J. Spec. Transl. 15, 142–162 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vander Wilt, D., Farbood, M.M. (2021). Deploying Prerecorded Audio Description for Musical Theater Using Live Performance Tracking. In: Kronland-Martinet, R., Ystad, S., Aramaki, M. (eds) Perception, Representations, Image, Sound, Music. CMMR 2019. Lecture Notes in Computer Science(), vol 12631. Springer, Cham. https://doi.org/10.1007/978-3-030-70210-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-70210-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-70209-0
Online ISBN: 978-3-030-70210-6
eBook Packages: Computer ScienceComputer Science (R0)