A Hidden Markov Model-Based Approach to Sequential Data Clustering

  • Antonello Panuccio
  • Manuele Bicego
  • Vittorio Murino
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2396)

Abstract

Clustering of sequential or temporal data is more challenging than traditional clustering as dynamic observations should be processed rather than static measures. This paper proposes a Hidden Markov Model (HMM)-based technique suitable for clustering of data sequences. The main aspect of the work is the use of a probabilistic model-based approach using HMM to derive new proximity distances, in the likelihood sense, between sequences. Moreover, a novel partitional clustering algorithm is designed which alleviates computational burden characterizing traditional hierarchical agglomerative approaches. Experimental results show that this approach provides an accurate clustering partition and the devised distance measures achieve good performance rates. The method is demonstrated on real world data sequences, i.e. the EEG signals due to their temporal complexity and the growing interest in the emerging field of Brain Computer Interfaces.

Keywords

Hide Markov Model Natural Cluster Hide Markov Model State Partitional Cluster Algorithm Discrete Hide Markov Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Rabiner, L. R.: A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proc. of IEEE 77(2) (1989) 257–286.CrossRefGoogle Scholar
  2. 2.
    Hu, J., Brown, M. K., Turin, W.: HMM based on-line handwriting recognition. IEEE Trans. Pattern Analysis and Machine Intelligence, 18(10) (1996) 1039–1045.CrossRefGoogle Scholar
  3. 3.
    Hughey, R., Krogh, A.: Hidden Markov Model for sequence analysis: extension and analysis of the basic method. Comp. Appl. in the Biosciences 12 (1996) 95–107.Google Scholar
  4. 4.
    Eickeler, S., Kosmala, A., Rigoll, G.: Hidden Markov Model based online gesture recognition. Proc. Int. Conf. on Pattern Recognition (ICPR) (1998) 1755–1757.Google Scholar
  5. 5.
    Jebara, T., Pentland, A.: Action Reaction Learning: Automatic Visual Analysis and Synthesis of interactive behavior. In 1st Intl. Conf. on Computer Vision Systems (ICVS’99) (1999).Google Scholar
  6. 6.
    Rabiner, L. R., Lee, C. H., Juang, B. H., Wilpon, J. G.: HMM Clustering for Connected Word Recognition. Proceedings of IEEE ICASSP (1989) 405–408.Google Scholar
  7. 7.
    Lee, K. F.: Context-Dependent Phonetic Hidden Markov Models for Speaker-Independent Continuous Speech Recognition. IEEE Transactions on Acoustics, Speech and Signal Processing 38(4) (1990) 599–609.CrossRefGoogle Scholar
  8. 8.
    Smyth, P.: Clustering sequences with HMM, in Advances in Neural Information Processing (M. Mozer, M. Jordan, and T. Petsche, eds.) MIT Press 9 (1997).Google Scholar
  9. 9.
    Li, C., Biswas, G.: Clustering Sequence Data using Hidden Markov Model Representation, SPIE’99 Conference on Data Mining and Knowledge Discovery: Theory, Tools, and Technology, (1999) 14–21.Google Scholar
  10. 10.
    Li, C., Biswas, G.: A Bayesian Approach to Temporal Data Clustering using Hidden Markov Models. Intl. Conference on Machine Learning (2000) 543–550.Google Scholar
  11. 11.
    Schwarz, G.: Estimating the dimension of a model. The Annals of Statistics, 6(2) (1978) 461–464.MATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    Stolcke, A., Omohundro, S.: Hidden Markov Model Induction by Bayesian Model Merging. Hanson, S. J., Cowan, J. D., Giles, C. L. eds. Advances in Neural Information Processing Systems 5 (1993) 11–18.Google Scholar
  13. 13.
    Cheeseman, P., Stutz, J.: Bayesian Classification (autoclass): Theory and Results. Advances in Knowledge discovery and data mining, (1996) 153–180.Google Scholar
  14. 14.
    Law, M. H., Kwok, J. T.: Rival penalized competitive learning for model-based sequence Proceedings Intl Conf. on Pattern Recognition (ICPR) 2 (2000) 195–198.Google Scholar
  15. 15.
    Penny, W. D., Roberts, S. J., Curran, E., Stokes, M.: EEG-based communication: a PR approach. IEEE Trans. Rehabilitation Engineering 8(2) (2000) 214–215.CrossRefGoogle Scholar
  16. 16.
    Juang, B. H., Levinson, S. E., Sondhi, M. M.: Maximum likelihood estimation for multivariate mixture observations of Markov Chain. IEEE Trans. Informat. Theory 32(2) (1986) 307–309.CrossRefGoogle Scholar
  17. 17.
    Juang, B. H., Rabiner, L. R.: Mixture autoregressive hidden Markov models for speech signals. IEEE Trans. Acoust. Speech Signal Proc. 33(6) (1985) 1404–1413.CrossRefMathSciNetGoogle Scholar
  18. 18.
    Penny, W. D., Roberts, S. J.: Dynamic models for nonstationary signal segmentation. Computers and Biomedical Research 32(6) (1998) 483–502.CrossRefGoogle Scholar
  19. 19.
    Kalman, R. E.: A New Approach to Linear Filtering and Prediction Problems. Transaction of the ASME-Journal of Basic Engineering (1960) 35–45.Google Scholar
  20. 20.
    Jazwinski, A.: Adaptive Filtering. Automatica 5 (1969) 475–485.MATHCrossRefGoogle Scholar
  21. 21.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press (1999).Google Scholar
  22. 22.
    Anderson, C. W., Stolz, E. A., Shamsunder, S.: Multivariate autoregressive models for classification of spontaneous electroencephalogram during mental tasks. IEEE Transactions on Biomedical Engineering, 45(3) (1998) 277–286.CrossRefGoogle Scholar
  23. 23.
    Nunez, P. L.: Neocortical Dynamics and Human EEG Rhythms. Oxford University Press, (1995).Google Scholar
  24. 24.
    Kaufman, L., Rousseuw, P.: Findings groups in Data: An Introduction to Cluster Analysis. John Wiley & Sons-New York (1990).Google Scholar
  25. 25.
    Keirn, Z.: Alternative modes of communication between man and machine. Master’s thesis. Purdue University (1988).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Antonello Panuccio
    • 1
  • Manuele Bicego
    • 1
  • Vittorio Murino
    • 1
  1. 1.Dipartimento di InformaticaUniversity of VeronaVeronaItaly

Personalised recommendations