Skip to main content
SpringerLink
Log in

The Entropy of a Binary Hidden Markov Process

  • Published:
Journal of Statistical Physics Aims and scope Submit manuscript

Abstract

The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter ε. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion coefficients up to second order in ε. Using a conjecture we extend the calculation to 11th order and discuss the convergence of the resulting series

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Y. Ephraim N. Merhav (2002) ArticleTitleHidden Markov processes IEEE Trans. Inform. Theory 48 1518–1569 Occurrence Handle10.1109/TIT.2002.1003838 Occurrence Handle2003f:94024

    Article  MathSciNet  Google Scholar 

  2. A. Schliep A. Schönhuth C. Steinhoff (2003) ArticleTitleUsing hidden Markov models to analyze gene expression time course data Bioinformatics 19 IssueIDSuppl. 1 i255–i263

    Google Scholar 

  3. L.R. Rabiner (1989) ArticleTitleA tutorial on hidden Markov models and selected applications in speech recognition Proc. IEEE 77 257–286 Occurrence Handle10.1109/5.18626

    Article  Google Scholar 

  4. I. Kanter A. Frydman A. Ater (2005) ArticleTitleIs a multiple excitation of a single atom equivalent to a single ensemble of atoms? Europhys. Lett. 69 IssueID6 874–878 Occurrence Handle10.1209/epl/i2004-10441-9

    Article  Google Scholar 

  5. I. Kanter A. Frydman A. Ater (2005) ArticleTitleUtilizing hidden Markov processes as a new tool for experimental physics Europhys. Lett. 69 IssueID5 798–804 Occurrence Handle10.1209/epl/i2004-10407-y

    Article  Google Scholar 

  6. Shannon C.E. A mathematical theory of communication. Bell Sys. Tech. J. 27:379–423 and 623–656 (1948).

    Google Scholar 

  7. Nattermann T., Theory of the random field Ising model, in Spin Glasses and Random Fields, Young A.P., ed. (World Scientific, 1997).

  8. Jacquet P., Seroussi G., and Szpankowski W., On the Entropy of a Hidden Markov Process, Data Compression Conference, Snowbird (2004).

  9. Ordentlich E., and Weissman T., New Bounds on the Entropy Rate of Hidden Markov Processes, San Antonio Information Theory Workshop (Oct. 2004).

  10. A preliminary presentation of our results is given in Zuk O., Kanter I., and Domany E., Asymptotics of the Entropy Rate for a Hidden Markov Process, Data Compression Conference, Snowbird (2005).

  11. T.M. Cover J.A. Thomas (1991) Elements of Information Theory Wiley New York

    Google Scholar 

  12. Saul L.K., and Jordan M.I., Boltzmann Chains and Hidden Markov Models, Advances in Neural Information Processing Systems, Volume 7 (MIT Press, 1994).

  13. D.J.C. MacKay (1996) ArticleTitleEquivalence of Boltzmann chains and Hidden Markov Models Neural Comput. 8 IssueID1 178–181 Occurrence Handle97i:81051

    MathSciNet  Google Scholar 

  14. B. Derrida M.M. France J. Peyriere (1986) ArticleTitleExactly solvable one-dimensional inhomogeneous models J. of Stat. Phys 45 IssueID3–4 439–449

    Google Scholar 

  15. D.S. Fisher P. Le Doussal P. Monthus (2001) ArticleTitleNonequilibrium dynamics of random field Ising spin chains: Exact results via real space renormalization group Phys. Rev. E 64 IssueID6 066–107 Occurrence Handle10.1103/PhysRevE.64.066107 Occurrence Handle2002k:82071

    Article  MathSciNet  Google Scholar 

  16. G. Grinstein D. Mukamel (1983) ArticleTitleExact solution of a one-dimensional ising-model in a random magnetic field Phys. Rev. B 27 4503–4506 Occurrence Handle10.1103/PhysRevB.27.4503 Occurrence Handle1983PhRvB..27.4503G

    Article  ADS  Google Scholar 

  17. Nieuwenhuizen T.M., and Luck J.M., Exactly soluble random field Ising models in one-dimension, J. Phys. A: Math. Gen. 19:1207–1227 (May 1986).

  18. Derrida B., and Hilhorst H.J. (1983). Singular behavior of certain infinite products of random 2× 2 matrices. J. Phys. A 16:2641–2654

    Google Scholar 

  19. http://www.maplesoft.com/

Download references

Author information

Authors and Affiliations

  1. Department of Physics of Complex Systems, Weizmann Institute of Science, Rehovot, 76100, Israel

    Or Zuk & Eytan Domany

  2. Department of Physics, Bar Ilan University, Ramat Gan, 52900, Israel

    Ido Kanter

Authors
  1. Or Zuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ido Kanter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eytan Domany
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eytan Domany.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zuk, O., Kanter, I. & Domany, E. The Entropy of a Binary Hidden Markov Process. J Stat Phys 121, 343–360 (2005). https://doi.org/10.1007/s10955-005-7576-y

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10955-005-7576-y

Keywords

Search

Navigation