Data-Driven Approaches to Objective Evaluation of Phoneme Alignment Systems
This paper presents techniques for objective characterisation of Automatic Speech-to-Phoneme Alignment (ASPA) systems, without the need for human-generated labels to act as a benchmark. As well as being immune to the effects of human variability, these techniques yield diagnostic information which can be helpful in the development of new alignment systems, ensuring that the resulting labels are as consistent as possible. To illustrate this, a total of 48 ASPA systems are used, including three front-end processors. For each processor, the number of states in each phoneme model, and of Gaussian distributions in each state mixture, are adjusted to generate a broad variety of systems. The results are compared using a statistical measure and a model-based Bayesian Monte-Carlo approach. The most consistent alignment system is identified, and is (as expected) in close agreement with typical “baseline” systems used in ASR research.
KeywordsPhonetic alignment label accuracy phoneme detectivity objective evaluation
Unable to display preview. Download preview PDF.
- 1.Baghai-Ravary, L.: Multi-dimensional Adaptive Signal Processing, with Application to Speech Recognition, Speech Coding and Image Compression. University of Sheffield PhD. Thesis (1995)Google Scholar
- 2.Beet, S.W., Gransden, I.R.: Interfacing an Auditory Model to a Parametric Speech Recogniser. Proc. Insititute of Acoustics 14(6), 321–328 (1992)Google Scholar
- 3.Chen, L., Liu, Y., Maia, E., Harper, M.: Evaluating Factors Impacting the Accuracy of Forced Alignments in a Multimodal Corpus. In: 4th International Conference on Language Resources and Evaluation (LREC), ELRA (2004)Google Scholar
- 6.Kochanski, G., Orphanidou, C.: Testing the Ecological Validity of Repetitive Speech. In: Proc. International Congress of Phonetic Sciences (ICPhS 2007), IPA (2007), http://www.icphs2007.de/conference/Papers/1632/1632.pdf
- 7.Kochanski, G., Rosner, B.S.: Bootstrap Markov Chain Monte Carlo and Optimal Solutions to The Law of Categorical Judgement (Corrected). Submitted to Behavior Research Methods (2010), http://arxiv.org/abs/1008.1596
- 8.Lander, T.: CSLU Labeling Guide, Center for Spoken Language Understanding, Oregon Graduate Institute (1997)Google Scholar
- 9.Ljolje, A., Riley, M.D.: Automatic Segmentation of Speech for TTS. In: Proc 3rd European Conference on Speech Communication and Technology (EUROSPEECH 1993), ESCA, pp. 1445–1448 (1993)Google Scholar
- 11.Sebestyen, G.S.: Decision-Making Processes in Pattern Recognition. ACM Monograph Series, pp. 40–47. MacMillan, Basingstoke (1962)Google Scholar
- 12.SoX Sound eXchange manual (2009), http://sox.sourceforge.net/sox.html
- 13.Young, S.J., et al.: The HTK Book (for HTK Version 3.4). Cambridge University Engineering Department (2009), http://htk.eng.cam.ac.uk/docs/docs.shtml