A Study on High-Order Hidden Markov Models and Applications to Speech Recognition

  • Lee-Min Lee
  • Jia-Chien Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4031)


We propose high-order hidden Markov models (HO-HMM) to capture the duration and dynamics of speech signal. In the proposed model, both the state transition probability and the output observation probability depend not only on the current state but also on several previous states. An extended Viterbi algorithm was developed to train model and recognize speech. The performance of the HO-HMM was investigated by conducting experiments on speaker independent Mandarin digits recognition. From the experimental results, we find that as the order of HO-HMM increases, the number of error reduces. We also find that systems with both high-order state transition probability distribution and output observation probability distribution outperform systems with only high-order state transition probability distribution.


Hide Markov Model Speech Recognition Automatic Speech Recognition State Transition Probability Maximum Likelihood Linear Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–285 (1991)CrossRefGoogle Scholar
  2. 2.
    Levinson, S.E.: Continuously variable duration hidden Markov models for automatic speech recognition. Computer Speech and Language 1(1), 29–45 (1986)CrossRefGoogle Scholar
  3. 3.
    Russell, M.J., Cook, A.: Experimental evaluation of duration modeling techniques for automatic speech recognition. In: Proc. IEEE ICASSP, pp. 2376–2379 (1987)Google Scholar
  4. 4.
    Furui, S.: Speaker independent isolated word recognition using dynamic features of speech spectrum. IEEE Trans. Acoust., Speech, Signal Processing, 52–59 (1986)Google Scholar
  5. 5.
    Atal, B.S.: Effectiveness of linear prediction characteristics of the speech wave for automatic speaker identification and verification. J. Acoust. Soc. Am. 55(6), 1304–1312 (1974)CrossRefGoogle Scholar
  6. 6.
    Gales, M.J.F.: Maximum Likelihood Linear Transformations for HMM-based Speech Recognition, Tech. Report, CUED/FINFENG/TR291, Cambridge Univ. (1997)Google Scholar
  7. 7.
    Bahl, L.R., de Souza, P.V., Gopalakrishnan, P.S., Nahamoo, D., Picheny, M.A.: Decision Trees for Phonological Rules in Continuous Speech. In: Proc. of the IEEE ICASSP, Toronto, Canada, pp. 185–188 (1991)Google Scholar
  8. 8.
    Mari, J.-F., Haton, J.-P., Kriouile, A.: Automatic word recognition based on second-order hidden Markov models. IEEE Transactions on Speech and Audio Processing 5(1), 22–25 (1997)CrossRefGoogle Scholar
  9. 9.
    du Preez, J.A.: Algorithms for high order hidden Markov modeling. In: Proceedings of the IEEE South African Symposium on Communications and Signal Processing, September 9-10, pp. 101–106 (1997)Google Scholar
  10. 10.
    Deng, L., Aksmanovic, M., Sun, D., Wu, C.F.J.: Speech recognition using hidden Markov models with polynomial regression functions as nonstationary states. IEEE Transactions on Speech and Audio Processing 2(4), 507–520 (1994)CrossRefGoogle Scholar
  11. 11.
    Linde, Y., Buzo, A., Gray, R.M.: An Algorithm for Vector Quantizer Design. IEEE Transactions on Communications, 702–710 (1980)Google Scholar
  12. 12.
    He, Y.: Extended Viterbi algorithm for second-order hidden Markov process. In: Proceedings of the IEEE 9th International Conference on Pattern Recognition, pp. 718–720 (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lee-Min Lee
    • 1
  • Jia-Chien Lee
    • 2
  1. 1.Department of Electrical EngineeringDa Yeh UniversityTaiwan
  2. 2.Department of Communication EngineeringDa Yeh UniversityTaiwan

Personalised recommendations