Abstract
In this paper, we illustrate the use of the MAGE performative speech synthesizer through its application to the conversion of realtime-measured facial features with FaceOSC into speech synthesis features such as vocal tract shape or intonation. MAGE is a new software library for using HMM-based speech synthesis in reactive programming environments. MAGE uses a rewritten version of the HTS engine enabling the computation of speech audio samples on a two-label window instead of the whole sentence. Only this feature enables the realtime mapping of facial attributes to synthesis parameters.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
d’Alessandro, N., Dutoit, T.: HandSketch Bi-Manual Controller: Investigation on Expressive Control Issues of an Augmented Tablet. In: Proc. International Conference on New Interfaces for Musical Expression, pp. 78–81 (2007)
MAGE and Face Tracking, https://vimeo.com/39567236
MAGE and HandSketch, https://vimeo.com/39558917
MAGE website, http://mage.numediart.org
Nordstorm, K., et al.: Developing Vowels Mappings for an Interactive Voice Synthesis System Controlled by Hand Motions. Journal of the Acoustical Society of America 127, 2021 (2010)
Zen, H., Tokuda, K., Black, A.: Statistical Parametric Speech Synthesis. Speech Communications 51(11), 1039–1064 (2009)
FaceOSC, https://vimeo.com/26098366
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
d’Alessandro, N., Astrinaki, M., Dutoit, T. (2013). MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters. In: Mancas, M., d’ Alessandro, N., Siebert, X., Gosselin, B., Valderrama, C., Dutoit, T. (eds) Intelligent Technologies for Interactive Entertainment. INTETAIN 2013. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 124. Springer, Cham. https://doi.org/10.1007/978-3-319-03892-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-03892-6_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03891-9
Online ISBN: 978-3-319-03892-6
eBook Packages: Computer ScienceComputer Science (R0)