Logic and Model Checking for Hidden Markov Models

  • Lijun Zhang
  • Holger Hermanns
  • David N. Jansen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3731)


The branching-time temporal logic PCTL* has been intro- duced to specify quantitative properties over probability systems, such as discrete-time Markov chains. Until now, however, no logics have been defined to specify properties over hidden Markov models (HMMs). In HMMs the states are hidden, and the hidden processes produce a se- quence of observations. In this paper we extend the logic PCTL* to POCTL*. With our logic one can state properties such as “there is at least a 90 percent probability that the model produces a given sequence of observations” over HMMs. Subsequently, we give model checking algorithms for POCTL* over HMMs.


  1. 1.
    Andova, S., Hermanns, H., Katoen, J.-P.: In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Baier, C.: On Algorithmic Verifiation Methods for Probabilistic Systems, Habilitations- schrift zur Erlangung der venia legendi der Fakultät für Mathematik and Informatik, Universität Mannheim (1998)Google Scholar
  3. 3.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Efficient computation of time-bounded reachability probabilities in uniform continuous-time markov decision processes. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 61–76. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Bianco, A., de Alfaro, L.: Model Checking of Probabilistic and Nondeterministic Systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  5. 5.
    Birney, E.: Hidden Markov models in biological sequence analysis. IBM Journal of Research and Development 45(3), 449–454 (2001)CrossRefGoogle Scholar
  6. 6.
    Courcoubetis, C., Yannakakis, M.: Verifying Temporal Properties of Finite- State Probabilistic Programs. In: FOCS, pp. 338–345. IEEE Computer Society Press, Los Alamitos (October 1988)Google Scholar
  7. 7.
    Courcoubetis, C., Yannakakis, M.: The Complexity of Probabilistic Verification. Journal of the ACM 42(4), 857–907 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, Technical report STAN-CS-TR-98-1601 (1997)Google Scholar
  9. 9.
    François, J.-M., Leduc, G.: Mobility prediction’s in°uence on QoS in wireless networks: A study on a call admission algorithm. In: 3rd International Symposium on Modeling and Optimization in Mobile, Ad-Hoc and Wireless Networks, pp. 238–247. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  10. 10.
    Gerth, R., Peled, D., Vardi, M.Y., Wolper, P.: Simple On-the-fly Automatic Verification of Linear Temporal Logic. In: PSTV, vol. 38, pp. 3–18. Chapman & Hall, Boca Raton (1995)Google Scholar
  11. 11.
    Hansson, H., Jonsson, B.: A Logic for Reasoning about Time and Reliability. Formal Aspects of Computing 6(5), 512–535 (1994)CrossRefzbMATHGoogle Scholar
  12. 12.
    Hauskrecht, M.: Value-Function Approximations for Partially Observable Markov Decision Processes. Journal of Artificial Intelligence Research 13, 33–94 (2000)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Jurafsky, D., Martin, J.H.: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice Hall, Englewood Cliffs (2000)Google Scholar
  14. 14.
    Nicola, R.D., Vaandrager, F.W.: Action versus state based logics for transition systems. In: Guessarian, I. (ed.) LITP 1990. LNCS, vol. 469, pp. 407–419. Springer, Heidelberg (1990)CrossRefGoogle Scholar
  15. 15.
    Pevzner, P.A.: Computational Molecular Biology: An Algorithmic Approach. The MIT Press, Cambridge (2000)zbMATHGoogle Scholar
  16. 16.
    Poupart, P.: Approximate Value-Directed Belief State Monitoring for Partially Observable Markov Decision Processes. Master’s thesis, University of British Columbia (November 2000)Google Scholar
  17. 17.
    Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  18. 18.
    Safra, S.: On the complexity ω-automata. In: FOCS, pp. 319–327 (1988)Google Scholar
  19. 19.
    Safra, S.: Exponential determinization ω-automata with strong-fairness accep- tance condition. In: STOC, pp. 275–282 (1992)Google Scholar
  20. 20.
    Salamatian, K., Vaton, S.: Hidden markov modeling for network communication channels. In: SIGMETRICS, pp. 92–101. ACM Press, New York (2001)Google Scholar
  21. 21.
    Vardi, M.Y., Wolper, P.: An Automata-Theoretic Approach to Automatic Program Verification. In: LICS, pp. 332–345. IEEE Computer Society Press, Los Alamitos (1986)Google Scholar
  22. 22.
    Vlontzos, J.A., Kung, S.Y.: Hidden Markov models for character recognition. IEEE Transactions on Image Processing 1, 539–543 (1992)CrossRefGoogle Scholar
  23. 23.
    Wolper, P., Vardi, M.Y., Sistla, A.P.: Reasoning about Infinite Computation Paths. In: FOCS 1983, pp. 185–194. IEEE Computer Society Press, Los Alamitos (1982)Google Scholar
  24. 24.
    Zhang, L., Hermanns, H., Jansen, D.N.: Logic and Model Checking for Hidden Markov Chais. AVACS Technical Report No. 6, SFB/TR 14 AVACS (May 2005) ISSN: 1860-9821,

Copyright information

© IFIP International Federation for Information Processing 2005

Authors and Affiliations

  • Lijun Zhang
    • 1
  • Holger Hermanns
    • 1
    • 2
  • David N. Jansen
    • 2
  1. 1.Department of Computer ScienceSaarland UniversitySaarbrückenGermany
  2. 2.Department of Computer ScienceUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations