Some Aspects of ASR Transcription Based Unsupervised Speaker Adaptation for HMM Speech Synthesis

  • Bálint Tóth
  • Tibor Fegyó
  • Géza Németh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6231)


Statistical parametric synthesis offers numerous techniques to create new voices. Speaker adaptation is one of the most exciting ones. However, it still requires high quality audio data with low signal to noise ration and precise labeling. This paper presents an automatic speech recognition based unsupervised adaptation method for Hidden Markov Model (HMM) speech synthesis and its quality evaluation. The adaptation technique automatically controls the number of phone mismatches. The evaluation involves eight different HMM voices, including supervised and unsupervised speaker adaptation. The effects of segmentation and linguistic labeling errors in adaptation data are also investigated. The results show that unsupervised adaptation can contribute to speeding up the creation of new HMM voices with comparable quality to supervised adaptation.


HMM-based speech synthesis unsupervised adaptation automatic speech recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Black, A., Zen, H., Tokuda, K.: Statistical Parametric Speech Synthesis. In: ICASSP 2007, pp. 1229–1232 (2007)Google Scholar
  2. 2.
    Iwahashi, N., Sagisaka, Y.: Speech Spectrum Conversion Based on Speaker Interpolation and Multi-Functional Representation with Weighting by Radial Basis Function Networks. Speech Communications 16(2), 139–151 (1995)CrossRefGoogle Scholar
  3. 3.
    Tachibana, M., Yamagishi, J., Masuko, T., Kobayashi, T.: Speech Synthesis with Various Emotional Expressions and Speaking Styles by Style Interpolation and Morphing. IEICE Trans. Inf. Syst. E88-D(11), 2484–2491 (2005)CrossRefGoogle Scholar
  4. 4.
    Tamura, M., Masuko, T., Tokuda, K., Kobayashi, T.: Adaptation of Pitch and Spectrum for HMM-Based Speech Synthesis Using MLLR. In: ICASSP 2001, pp. 805–808 (1998)Google Scholar
  5. 5.
    Ogata, K., Tachibana, M., Yamagishi, J., Kobayashi, T.: Acoustic Model Training Based on Linear Transformation and MAP Modification for HSMM-Based Speech Synthesis. In: ICSLP 2006, pp. 1328–1331 (2006)Google Scholar
  6. 6.
    Kawai, H., Toda, T., Ni, J., Tsuzaki, M., Tokuda, K.: XIMERA: A New TTS from ATR Based on Corpus-Based Technologies. In: ISCA SSW5 2004, pp. 179–184 (2004)Google Scholar
  7. 7.
    Plumpe, M., Acero, A., Hon, H.-W., Huang, X.-D.: HMM-Based Smoothing for Concatenative Speech Synthesis. In: ICSLP 1998, pp. 2751–2754 (1998)Google Scholar
  8. 8.
    Okubo, T., Mochizuki, R., Kobayashi, T.: Hybrid Voice Conversion of Unit Selection and Generation using Prosody Dependent HMM. IEICE Trans. Inf. Syst. E89-D(11), 2775–2782 (2006)CrossRefGoogle Scholar
  9. 9.
    Mihajlik, P., Fegyó, T., Tüske Z., Ircing, P.: A Morpho-graphemic Approach for the Recognition of Spontaneous Speech in Agglutinative Languages like Hungarian. In: Interspeech 2007, pp. 1497–1500 (2007)Google Scholar
  10. 10.
    King, S., Tokuda, K., Zen, H., Yamagishi, J.: Unsupervised Adaptation for HMM-Based Speech Synthesis. In: Interspeech 2008, pp. 1869–1872 (2008)Google Scholar
  11. 11.
    Gibson, M.: Two-Pass Decision Tree Construction for Unsupervised Adaptation of HMM-Based Synthesis Models. In: Interspeech 2009, pp. 1791–1794 (2009)Google Scholar
  12. 12.
    Yamagishi, J., Ling, Z., King, S.: Robustness of HMM-Based Speech Synthesis. In: Interspeech 2008, pp. 581–584 (2008)Google Scholar
  13. 13.
    Tóth, B., Németh, G.: Hidden Markov Model Based Speech Synthesis System in Hungarian. Infocommunications Journal LXIII(2008/7), 30–34 (2008)Google Scholar
  14. 14.
    Mihajlik, P., Tarján, B., Tüske, Z., Fegyó, T.: Investigation of Morph-based Speech Recognition Improvements across Speech Genres In: Interspeech 2009, pp. 2687–2690 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Bálint Tóth
    • 1
  • Tibor Fegyó
    • 1
  • Géza Németh
    • 1
  1. 1.Department of Telecommunications and Media InformaticsBudapest University of Technology and Economics 

Personalised recommendations