State Level Control for Acoustic Model Training
We propose a method for controlling Gaussian mixture splitting in HMM states during the training of acoustic models. The method is based on introducing special criteria of mixture quality in every state. These criteria are calculated over a separate part of the speech database. We back up states before splitting and revert to saved copies of the states whose criteria values have decreased, which makes it possible to optimize the number of Gaussians in the GMMs of the states and to prevent overfitting. The models obtained by such training demonstrate improved recognition rate with a significantly smaller number of Gaussians per state.
Keywordsacoustic modeling GMM-HMM models Gaussian splitting
Unable to display preview. Download preview PDF.
- 4.Giuliani, D., De Mori, R.: Speaker adaptation, pp. 363–404. Academic Press Inc., London (1998) R. De Mori ed.Google Scholar
- 5.Gales, M.J.F., Young, S.J.: HMM recognition in noise using parallel model combination. In: Proc. of the EuroSpeech, Berlin, Germany, September 22-25, pp. 837–840 (1993)Google Scholar
- 6.Mohri, M., Pereira, F., Riley, M.: Speech Recognition with Weighted Finite-State Transducers. In: Benesty, J., Sondhi, M.M., Huang, Y.A. (eds.) Springer Handbook of Speech Processing, pp. 559–584. Springer (2008)Google Scholar
- 7.Bilmes, J.A.: A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models. Technical report TR-97-021, International Computer Science Institute, Berkley (1998)Google Scholar
- 8.Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. J. Royal Statistical Society, Series B, 39(1), 1–38 (1977)Google Scholar
- 9.Steele, R.J., Raftery, A.E.: Performance of Bayesian Model Selection Criteria for Gaussian Mixture Models. Technical report TR-559, University of Washington, Dept. of Statistics (2009)Google Scholar
- 10.Odell, J.J.: The Use of Context in Large Vocabulary Speech Recognition (PhD Thesis), Cambridge: Cambridge University (1995)Google Scholar
- 11.Tatarnikova, M., Tampel, I., Oparin, I., Khokhlov, Y.: Building Acoustic Models for Large Vocabulary Continuous Speech Recognizer for Russian. In: Proc. of the SpeCom, St. Petersburg, Russia, June 26-29, pp. 83–87 (2006)Google Scholar
- 12.Povey, D.: Discriminative training for large vocabulary speech recognition (PhD thesis), Cambridge: Cambridge University (2003)Google Scholar
- 13.Yurkov, P., Korenevsky, M., Levin, K.: An Improvement of Robustness to Speech Loudness Change for an ASR System Based on LC-RC Features. In: Proc. of the SpeCom., Kazan, Russia, September 27-30, pp. 62–66 (2011)Google Scholar
- 14.Van den Heuvel, H., et al.: SpeechDat-E: Five Eastern European Speech Databases for Voice-Operated Teleservices Completed. In: Proc. of the InterSpeech, Aalborg, Denmark, September 3-7, pp. 2059–2062 (2001)Google Scholar