A Continuous Vocoder Using Sinusoidal Model for Statistical Parametric Speech Synthesis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11096)


In our earlier work in statistical parametric speech synthesis, we proposed a source-filter based vocoder using continuous F0 (contF0) in combination with Maximum Voiced Frequency (MVF), which was successfully used with deep learning. The advantage of a continuous vocoder in this scenario is that vocoder parameters are simpler to model than conventional vocoders with discontinuous F0. However, our vocoder lacks some degree of naturalness and still not achieving a high-quality speech synthesis compared to the well-known vocoders (e.g. STRAIGHT or WORLD). Previous studies have shown that human voice can be modelled effectively as a sum of sinusoids. In this paper, we firstly address the design of a continuous vocoder using sinusoidal synthesis model that is applicable in statistical frameworks. The same three parameters of the analysis part from our previous model have been also extracted and used for this study. For refining the output of the contF0 estimation, post-processing approach is utilized to reduce the unwanted voiced component of unvoiced speech sounds, resulting in a smoother contF0 track. During synthesis, a sinusoidal model with minimum phase is applied to reconstruct speech. Finally, we have compared the voice quality of the proposed system to the STRAIGHT and WORLD vocoders. Experimental results from objective and subjective evaluations have shown that the proposed vocoder gives state-of-the-art vocoders performance in synthesized speech while outperforming the previous work of our continuous F0 based source-filter vocoder.


Continuous vocoder Speech synthesis Sinusoidal model  ContF0 



The research was partly supported by the VUK (AAL-2014-1-183), and by the EUREKA (DANSPLAT E!9944) projects. The Titan X GPU used for this research was donated by NVIDIA Corporation. We would like to thank the subjects for participating in the listening test.


  1. 1.
    Zen, H., Tokuda, K., Black, A.: Statistical parameteric speech synthesis. Speech Commun. 51(11), 1039–1064 (2009)CrossRefGoogle Scholar
  2. 2.
    Kawahara, H., Masuda-Katsuse, I., de Cheveign, A.: Restructuring speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: possible role of a repetitive structure in sounds. Speech Commun. 27(3), 187–207 (1999)CrossRefGoogle Scholar
  3. 3.
    Drugman, T., Wilfart, G., Dutoit, T.: A deterministic plus stochastic model of the residual signal for improved parametric speech synthesis. In: 10th Proceedings of Interspeech, Brighton, pp. 1779–1782 (2009)Google Scholar
  4. 4.
    Morise, M., Yokomori, F., Ozawa, K.: WORLD: a vocoder-based high-quality speech synthesis system for real-time applications. IEICE Trans. Inf. Syst. 7(E99-D), 1877–1884 (2016)CrossRefGoogle Scholar
  5. 5.
    Csapó, T.G., Németh, G., Cernak, M.: Residual-based excitation with continuous F0 modeling in HMM-based speech synthesis. In: Dediu, A.-H., Martín-Vide, C., Vicsi, K. (eds.) SLSP 2015. LNCS (LNAI), vol. 9449, pp. 27–38. Springer, Cham (2015). Scholar
  6. 6.
    Garner, P.N., Cernak, M., Motlicek, P.: A simple continuous pitch estimation algorithm. IEEE Signal Process. Lett. 20(1), 102–105 (2013)CrossRefGoogle Scholar
  7. 7.
    Drugman, T., Stylianou, Y.: Maximum voiced frequency estimation: exploiting amplitude and phase spectra. IEEE Signal Process. Lett. 21(10), 1230–1234 (2014)CrossRefGoogle Scholar
  8. 8.
    Al-Radhi, M.S., Csapó, T.G., Németh, G.: Time-domain envelope modulating the noise component of excitation in a continuous residual-based vocoder for statistical parametric speech synthesis. In: 18th Proceedings of Interspeech, Stockholm, pp. 434–438 (2017)Google Scholar
  9. 9.
    McAulay, R.J., Quatieri, T.F.: Speech analysis/synthesis based on a sinusoidal representation. IEEE Trans. Acoust. Speech Signal Process. 34(4), 744–754 (1986)CrossRefGoogle Scholar
  10. 10.
    Stylianou, Y., Laroche, J., Moulines, E.: High-quality speech modification based on a harmonic + noise model. In: Proceedings of Eurospeech, Madrid, pp. 451–454 (1995)Google Scholar
  11. 11.
    Degottex, G., Stylianou, Y.: A full-band adaptive harmonic representation of speech. In: 13th Proceedings of Interspeech, Portland (2012)Google Scholar
  12. 12.
    Hu, Q., Stylianou, Y., Maia, R., Richmond, K., Yamagishi, J.: Methods for applying dynamic sinusoidal models to statistical parametric speech synthesis. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, pp. 4889–4893. IEEE (2015)Google Scholar
  13. 13.
    Tokuda, K., Kobayashi, T., Masuko, T., Imai, S.: Mel-generalized cepstral analysis - a unified approach to speech spectral estimation. In: 8th International Conference on Spoken Language Processing (ICSLP), Yokohama, pp. 1043–1046 (1994)Google Scholar
  14. 14.
    Imai, S., Sumita, K., Furuichi, C.: Mel Log Spectrum Approximation (MLSA) filter for speech synthesis. Electron. Commun. Jpn. (Part I: Communications) 66(2), 10–18 (1983)CrossRefGoogle Scholar
  15. 15.
    Talkin, D.: A robust algorithm for pitch tracking (RAPT). In: Kleijn, B., Paliwal, K. (eds.) Speech Coding and Synthesis, pp. 497–518. Elsevier, Amesterdam (1995)Google Scholar
  16. 16.
    Rabiner, L., Sambur, M., Schmidt, C.: Applications of a nonlinear smoothing algorithm to speech processing. IEEE Trans. Acoust. Speech Signal Process. 23(6), 552–557 (1975)CrossRefGoogle Scholar
  17. 17.
    Erro, D., Sainz, I., Navas, E., Hernaez, I.: Harmonics plus noise model based vocoder for statistical parametric speech synthesis. IEEE J. Sel. Top. Signal Process. 8(2), 184–194 (2014)CrossRefGoogle Scholar
  18. 18.
    Kominek, J., Black, A.W.: CMU ARCTIC databases for speech synthesis. Language Technologies Institute (2003).
  19. 19.
    Kawahara, H., Morise, M.: Technical foundations of TANDEM-STRAIGHT, a speech analysis, modification and synthesis framework. Sadhana 36(5), 713–727 (2011)CrossRefGoogle Scholar
  20. 20.
    Ma, J., Hu, Y., Loizou, P.: Objective measures for predicting speech intelligibility in noisy conditions based on new band-importance functions. Acoust. Soc. Am. 125(5), 3387–3405 (2009)CrossRefGoogle Scholar
  21. 21.
    Jensen, J., Taal, C.H.: An algorithm for predicting the intelligibility of speech masked by modulated noise maskers. IEEE/ACM Trans. Audio Speech Lang. Process. 24(11), 2009–2022 (2016)CrossRefGoogle Scholar
  22. 22.
    Itakura, F., Saito, S.: An analysis-synthesis telephony based on the maximum-likelihood method. In: Proceedings of the 6th International Congress on Acoustics, Tokyo, pp. C17–C20 (1968)Google Scholar
  23. 23.
    Chen, J., Benesty, J., Huang, Y., Doclo, S.: New insights into the noise reduction Wiener filter. IEEE Trans. Audio Speech Lang. Process. 14(4), 1218–1234 (2006)CrossRefGoogle Scholar
  24. 24.
    ITU-R Recommendation BS.1534. Method for the subjective assessment of intermediate audio quality (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Telecommunications and Media InformaticsBudapest University of Technology and EconomicsBudapestHungary
  2. 2.MTA-ELTE Lendület Lingual Articulation Research GroupBudapestHungary

Personalised recommendations