Advertisement

Hidden Markov Models for Individual Time Series

  • Ingmar VisserEmail author
  • Maartje E. J. Raijmakers
  • Han L. J. van der Maas
Chapter
First Online:

Abstract

This chapter introduces hidden Markov models to study and characterize (individual) time series such as observed in psychological experiments of learning, repeated panel data, repeated observations comprising a developmental trajectory etc. Markov models form a broad and flexible class of models with many possible extensions, while at the same time allowing for relatively easy analysis and straightforward interpretation. Here we focus on hidden Markov models with a discrete underlying state space, and observations at discrete times; however, hidden Markov models are not limited to these situations and some pointers are provided to literature on possible extensions.

Download chapter PDF

This chapter introduces hidden Markov models to study and characterize (individual) time series such as observed in psychological experiments of learning, repeated panel data, repeated observations comprising a developmental trajectory etc. Markov models form a broad and flexible class of models with many possible extensions, while at the same time allowing for relatively easy analysis and straightforward interpretation. Here we focus on hidden Markov models with a discrete underlying state space, and observations at discrete times; however, hidden Markov models are not limited to these situations and some pointers are provided to literature on possible extensions.

Markov models have a long history in the social sciences; in psychology, for example, Markov models have been applied in analyzing language (Miller, 1952; Miller & Chomsky, 1963), in describing learning processes in paired associate learning (see Wickens, 1982, for an overview of models and techniques); in sociology, applications are mainly in the analysis of repeated measures of panel data (Langeheine & Van de Pol, 1990); similarly in political science (McCutcheon, 1987). Recently, extensions of Markov models, such as the hidden Markov model, have become increasingly popular, notably in speech recognition (Rabiner, 1989); in biology, in analyzing DNA sequences (Krogh, 1998); in econometric science, in analyzing changes in stock market prices and commodities (Kim, 1994); and finally, in machine learning and data mining (Ghahramani & Jordan, 1997). This chapter focusses on time series data from a psychological experiment in which both speed, i.e., reaction times, and accuracy are modeled simultaneously.

The rest of this chapter is organized as follows: In the next Section hidden Markov models are introduced in a conceptual fashion, and its relationship with other models is described. Following that, in Section Likelihood, Parameter Estimation, and Inference, the main characteristics of the likelihood function, parameter optimization and inference are discussed, thereby introducing the hidden Markov in a more formal way. The next Section discusses analyses of two real life data sets thereby illustrating various characteristics of hidden Markov models and their potential to deal with individual time series. We end by summarizing and discussing the main results.

Notes

Acknowledgement

Ingmar Visser and Maartje Raijmakers were supported by an EC Framework 6 grant, project 516542 (NEST). Thanks to the Water Corporation of Western Australia for providing us with the Perth water dams data.

References

  1. Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken, NJ: Wiley-Interscience.CrossRefGoogle Scholar
  2. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Csaki (Eds.), Second international symposium on information theory (pp. 267 281). Budapest: Academiai Kiado.Google Scholar
  3. Altman, R. M. (2004). Assessing the goodness-of-fit of hidden Markov models. Biometrics, 60, 444 450.CrossRefGoogle Scholar
  4. Ashby, F. G., & Ell, S. W. (2001). The neurobiology of human category learning. TRENDS in Cognitive Sciences, 5, 204 210.CrossRefPubMedGoogle Scholar
  5. Baum, L. E., & Petrie, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. Annals of Mathematical Statistics, 37, 1554 1563.CrossRefGoogle Scholar
  6. Böckenholt, U. (2005). A latent Markov model for the analysis of longitudinal datacollected in continuous time: States, durations, and transitions. Psychological Methods, 10(1), 65 83.CrossRefPubMedGoogle Scholar
  7. Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605 634.CrossRefPubMedGoogle Scholar
  8. Bureau, A., Shiboski, S., & Hughes, J. P. (2003). Applications of continuous time hidden Markov models to the study of misclassiffied disease outcomes. Statistics in Medicine, 22, 441 462.CrossRefPubMedGoogle Scholar
  9. Chung, H., Walls, T. A., & Park, Y. (2007). A latent transition model with logistic regression. Psychometrika, 72(3), 413 435.CrossRefGoogle Scholar
  10. Collins, L. M., & Wugalter, S. E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27(1), 131 157.CrossRefGoogle Scholar
  11. Eid, M., & Langeheine, R. (2003). Separating stable from variable individuals in longitudinal studies by mixture distribution models. Measurement: Interdisciplinary Research and Perspectives, 1(3), 179 206.CrossRefGoogle Scholar
  12. Fahrmeir, L., Tutz, G., & Hennevogl, W. (2001). Multivariate statistical modelling based on generalized linear models. New York: Springer.Google Scholar
  13. Ghahramani, Z., & Jordan, M. I. (1997). Factorial hidden Markov models. Machine Learning, 29, 245 273.CrossRefGoogle Scholar
  14. Jansen, B. R. J., Raijmakers, M. E. J., & Visser, I. (2007). Rule transition on the balance scale task: A case study in belief change. Synthese, 155(2), 211 236.CrossRefGoogle Scholar
  15. Kaplan, D. (2008). An overview of Markov chain methods for the study of stage-sequential developmental processes. Developmental Psychology, 44(2), 457 467.CrossRefGoogle Scholar
  16. Kim, C.-J. (1994). Dynamic linear models with Markov-switching. Journal of Econometrics, 60, 1 22.CrossRefGoogle Scholar
  17. Krogh, A. (1998). An introduction to hidden Markov models for biological sequences. In S. L. Salzberg, D. B. Searls, & S. Kasif (Eds.), Computational methods in molecular biology (pp. 45 63). Amsterdam: Elsevier.CrossRefGoogle Scholar
  18. Laming, D. R. J. (1968). Information theory of choice reaction times. New York: Academic Press.Google Scholar
  19. Langeheine, R., & Van de Pol, F. (1990). A unifying framework for Markov modeling in discrete space and discrete time. Sociological Methods and Research, 18(4), 416 441.CrossRefGoogle Scholar
  20. Langeheine, R., & Van de Pol, F. (2000). Fitting higher order Markov chains. Methods of Psychological Research Online, 5(1), 32 55.Google Scholar
  21. Lystig, T. C., & Hughes, J. P. (2002). Exact computation of the observed information matrix for hidden Markov models. Journal of Computational and Graphical Statistics, 11, 678 689.CrossRefGoogle Scholar
  22. McCutcheon, A. L. (1987). Latent class analysis (No. 07-064). Beverly Hills: Sage Publications.Google Scholar
  23. Miller, G. A. (1952). Finite Markov processes in psychology. Psychometrika, 17, 149 167.CrossRefGoogle Scholar
  24. Miller, G. A., & Chomsky, N. (1963). Finitary models of language users (chap. 13). In R. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology. New York: Wiley.Google Scholar
  25. Neale, M. C., Boker, S. M., Xie, G., & Maes, H. H. (2003). Mx: Statistical modeling. VCU Box 900126, Richmond, VA 23298.Google Scholar
  26. R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria (ISBN 3-900051-07-0).Google Scholar
  27. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of IEEE, 77(2), 267 295.CrossRefGoogle Scholar
  28. Raijmakers, M. E. J., Dolan, C. V., & Molenaar, P. C. M. (2001). Finite mixture distribution models of simple discrimination learning. Memory & Cognition, 29(5), 659 677.CrossRefGoogle Scholar
  29. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59 108.CrossRefGoogle Scholar
  30. Reboussin, B. A., Reboussin, D. M., Liang, K.-Y., & Anthony, J. C. (1998). Latent transition modeling of progression of health-risk behavior. Multivariate Behavioral Research, 33(4), 457 478.CrossRefGoogle Scholar
  31. Schmittmann, V. D., Visser, I., & Raijmakers, M. E. J. (2006). Multiple learning modes in the development of rule-based category-learning task performance. Neuropsychologia, 44(11), 2079 2091.CrossRefPubMedGoogle Scholar
  32. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461 464.CrossRefGoogle Scholar
  33. Tamura, R. (2007). Rdonlp2: An R extension library to use Peter Spelluci s DONLP2 from R (version 0.3-1) [R package].Google Scholar
  34. van der Maas, H. L. J., Dutilh, G., Visser, I., Grasman, R. P. P. P., & Wagenmakers, E.-J. (2008). Phase transitions in the trade-off between speed and accuracy in choice reaction time tasks. Manuscript in preparation.Google Scholar
  35. van der Maas, H. L. J., & Molenaar, P. (1992). Stagewise cognitive development: An application of catastrophe theory. Psychological Review, 99, 395 417. (Marine and atmospheric research. (n.d.). http://www.cmar.csiro.au/)CrossRefPubMedGoogle Scholar
  36. Visser, I. (2008). Depmix: An R-package for fitting mixtures of latent Markov models on mixed data with covariates (version 0.9.4) [R package]. Available at CRAN: http://cran.r-project.org
  37. Visser, I., Schmittmann, V. D., & Raijmakers, M. E. J. (2007). Markov process models for discrimination learning. In K. van Montfort, H. Oud, & A. Satorra (Eds.), Longitudinal models in the behavioral and related sciences (pp. 337 365). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  38. Visser, I., & Speekenbrink, M. (2008). DepmixS4: S4 Classes for hidden Markov Models [R package latest version]. Available at CRAN: http://cran.r-project.org
  39. Wald, A. (1943). Test of statistical hypotheses concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54, 426 482.CrossRefGoogle Scholar
  40. Water Corporation of Western Australia. http://www.watercorporation.com.au/
  41. Wickens, T. D. (1982). Models for behavior: Stochastic processes in psychology. San Francisco: W. H. Freeman and Company.Google Scholar
  42. Wickens, T. D. (1989). Multiway contingency tables analysis for the social sciences. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Ingmar Visser
    • 1
    Email author
  • Maartje E. J. Raijmakers
  • Han L. J. van der Maas
  1. 1.Developmental PsychologyUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations

Cite chapter