Pattern Analysis and Applications

, Volume 8, Issue 1–2, pp 115–124 | Cite as

Hidden Markov chain estimation and parameterisation via ICA-based feature-selection

  • David WindridgeEmail author
  • Richard Bowden


We set out a methodology for the automated generation of hidden Markov models (HMMs) of observed feature-space transitions in a noisy experimental environment that is maximally generalising under the assumed experimental constraints. Specifically, we provide an ICA-based feature-selection technique for determining the number, and the transition sequence of the underlying hidden states, along with the observed-state emission characteristics when the specified noise model assumptions are fulfilled. In retaining correlation information between features, the method is potentially more general than the commonly employed Gaussian mixture model HMM parameterisation methods, to which we demonstrate that our method reduces when an arbitrary separation of features, or an experimentally-limited feature-space is imposed. A practical demonstration of the application of this method to automated sign-language classification is given, for which we demonstrate that a performance improvement of the order of 40% over naive Markovian modelling of the observed transitions is possible.


Hidden Markov models ICA Feature-selection Parameterisation Feature-spaces Sign-recognition 



This paper gratefully acknowledges the financial support of INRIA within the context of the Cognitive Vision Systems framework collaboration.


  1. 1.
    Young S, Odell J, Ollason D, Valtchev V, Woodland P (1997) The HTK Book for HTK Version 2.1. Cambridge UniversityGoogle Scholar
  2. 2.
    McCane B, Caelli T (2004) Diagnostic tools for evaluating and updating hidden markov models. Pattern Recognition 37(7):1325–1337CrossRefGoogle Scholar
  3. 3.
    Brand M, Kettnaker V (2000) Discovery and segmentation of activities in video. IEEE Trans Pattern Anal Mach Intell 22(8):844–851CrossRefGoogle Scholar
  4. 4.
    Ash RB (1965) Information theory. Dover Publications, New YorkGoogle Scholar
  5. 5.
    Bahlmann C, Burkhardt H (2001) Measuring HMM similarity with the bayes probability of error and its application to online handwriting recognition. In: Proceedings of Sixth International Conference on Document Analysis and Recognition. Seatle, pp 406–411Google Scholar
  6. 6.
    Lyngso RB, Pedersen CNS, Nielsen H (1999) Metrics and similarity measures for hidden Markov models. In: Proceedings of ISMB, pp 178–186Google Scholar
  7. 7.
    Lyngso RB (2001) Pedersen, Complexity of Comparing Hidden Markov Models. In: Proceedings of the 12th International Symposium on Algorithms and Computation (ISAAC 2001), volume 2223 of Lecture Notes in Computer Science. Springer Verlag, pp 416–428Google Scholar
  8. 8.
    Rao AV, Rose K (1999) Deterministically annealed design of hidden Markov model speech recognizers. Submitted to IEEE Trans on Speech and Audio ProcessingGoogle Scholar
  9. 9.
    Balasubramanian V (1993) Equivalence and reduction of hidden markovmodels. M.I.T. Technical Report, AITR-1370Google Scholar
  10. 10.
    Brand ME (1999) Structure learning in conditional probability models via an eutropic prior and parameter extinction. Neural Computation J 11(5):1155–1182CrossRefGoogle Scholar
  11. 11.
    Lee KF (1989) Automatic speech recognition: the development of the SPHINX SYSTEM. Kluwer Academic Publishers, BostonGoogle Scholar
  12. 12.
    Ghahramani Z, Jordan MI (1996) Factorial hidden Markov models. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) NIPS, vol 8. CambridgeGoogle Scholar
  13. 13.
    Baum L, Eagon J (1967) An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology. Am Mathematical Soc Bull 73:360–363Google Scholar
  14. 14.
    Hyvrinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430CrossRefPubMedGoogle Scholar
  15. 15.
    Comon P (1994) Independent component analysis, A new concept? IEEE Signal Processing Mag 36(3)Google Scholar
  16. 16.
    Devijver PA, Kittler J (1982) Pattern recognition: a statistical approach. Prentice-HallGoogle Scholar
  17. 17.
    Asymptotics of graphical projection pursuit. In: Diaconis P, Freedman D. Ann Stat 12:793–815Google Scholar
  18. 18.
    Press WH, Flannery BP, Teukolsky SA, Vetterling WT (1992) Jacobi transformation of a symmetric matrix. 11.1 In: numerical recipes in FORTRAN: The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge England, pp 456–462Google Scholar
  19. 19.
    Vogler C, Metaxas D (1999) Towards Scalability in ASL Recognition: Breaking Down Signs into Phonemes. Gesture Workshop’99, March 17–19, Gif-sur-Yvette, FranceGoogle Scholar
  20. 20.
    Smith C (1990) Signs make sense: a guide to british sign language. Souvenir Press Ltd, ISBN 0285650831Google Scholar
  21. 21.
    Sturges J, Whitfield T (1995) Locating basic colours in the munsell space. Color Res Appl 20(6):364Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Authors and Affiliations

  1. 1.Department of Electronic and Electrical EngineeringUniversity of SurreyGuildfordUK

Personalised recommendations