Belief Bisimulation for Hidden Markov Models

Logical Characterisation and Decision Algorithm
  • David N. Jansen
  • Flemming Nielson
  • Lijun Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7226)


This paper establishes connections between logical equivalences and bisimulation relations for hidden Markov models (HMM). Both standard and belief state bisimulations are considered.

We also present decision algorithms for the bisimilarities. For standard bisimilarity, an extension of the usual partition refinement algorithm is enough. Belief bisimilarity, being a relation on the continuous space of belief states, cannot be described directly. Instead, we show how to generate a linear equation system in time cubic in the number of states.


Markov Chain Hide Markov Model Model Check Belief State Decision Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baier, C., Katoen, J.-P.: Principles of model checking. MIT Press, Cambridge (2008)zbMATHGoogle Scholar
  2. 2.
    Baier, C., Katoen, J.-P., Hermanns, H., Wolf, V.: Comparative branching-time semantics for Markov chains. Inf. Comput. 200(2), 149–214 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bicego, M., Dovier, A., Murino, V.: Designing the Minimal Structure of Hidden Markov Model by Bisimulation. In: Figueiredo, M., Zerubia, J., Jain, A.K. (eds.) EMMCVPR 2001. LNCS, vol. 2134, pp. 75–90. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  4. 4.
    Castro, P.S., Panangaden, P., Precup, D.: Equivalence relations in fully and partially observable Markov decision processes. In: Boutilier, C. (ed.) Proc. of the Twenty-First Intl. Joint Conference on Artificial Intelligence, IJCAI-2009, pp. 1653–1658. AAAI Press, Menlo Park (2009)Google Scholar
  5. 5.
    Christiansen, H., Have, C.T., Lassen, O.T., Petit, M.: Inference with constrained hidden Markov models in PRISM. TPLP 10(4-6), 449–464 (2010)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Derisavi, S., Hermanns, H., Sanders, W.H.: Optimal state-space lumping in Markov chains. Inf. Proc. Lett. 87(6), 309–315 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Doyen, L., Henzinger, T.A., Raskin, J.F.: Equivalence of labeled Markov chains. Int. J. Foundations of Computer Science 19(3), 549–563 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Asp. Comput. 6(5), 512–535 (1994)zbMATHCrossRefGoogle Scholar
  9. 9.
    Jansen, D.N., Nielson, F., Zhang, L.: Belief bisimulation for hidden Markov models: logical characterisation and decision algorithm. Tech. Rep. ICIS-R12002, Radboud Universiteit: ICIS, Nijmegen (2012)Google Scholar
  10. 10.
    Jurafsky, D., Martin, J.H.: Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. Prentice Hall, Upper Saddle River (2000)Google Scholar
  11. 11.
    Katoen, J.-P., Kemna, T., Zapreev, I., Jansen, D.N.: Bisimulation Minimisation Mostly Speeds Up Probabilistic Model Checking. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 87–101. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Kiefer, S., Murawski, A.S., Ouaknine, J., Wachter, B., Worrell, J.: Language Equivalence for Probabilistic Automata. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 526–540. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. Inf. Comput. 94(1), 1–28 (1991)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Paige, R., Tarjan, R.E.: Three partition refinement algorithms. SIAM J. Comput. 16(6), 973–989 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  16. 16.
    Salamatian, K., Vaton, S.: Hidden Markov modeling for network communication channels. In: Proc. 2001 ACM SIGMETRICS Intl. Conf. on Measurement and Modeling of Computer Systems, vol. 29, pp. 92–101. ACM, New York (2001)CrossRefGoogle Scholar
  17. 17.
    Sistla, A.P., Žefran, M., Feng, Y.: Monitorability of Stochastic Dynamical Systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 720–736. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  18. 18.
    Stein, E.M., Shakarchi, R.: Real analysis: measure theory, integration, and Hilbert spaces, Princeton lectures in analysis, vol. III. Princeton Univ. Pr., Princeton (2005)Google Scholar
  19. 19.
    Woodward, A., Penn, R.: The wrong kind of snow. Hodder & Stoughton, London (2007)Google Scholar
  20. 20.
    Zhang, L., Hermanns, H., Jansen, D.N.: Logic and Model Checking for Hidden Markov Models. In: Wang, F. (ed.) FORTE 2005. LNCS, vol. 3731, pp. 98–112. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • David N. Jansen
    • 1
  • Flemming Nielson
    • 2
  • Lijun Zhang
    • 2
  1. 1.Radboud Universiteit NijmegenThe Netherlands
  2. 2.Technical University of DenmarkDenmark

Personalised recommendations