Skip to main content

Permutation approach to finite-alphabet stationary stochastic processes based on the duality between values and orderings

The European Physical Journal Special Topics volume 222, pages 383–399 (2013)Cite this article

Abstract

The duality between values and orderings is a powerful tool to discuss relationships between various information-theoretic measures and their permutation analogues for discrete-time finite-alphabet stationary stochastic processes (SSPs). Applying it to output processes of hidden Markov models with ergodic internal processes, we have shown in our previous work that the excess entropy and the transfer entropy rate coincide with their permutation analogues. In this paper, we discuss two permutation characterizations of the two measures for general ergodic SSPs not necessarily having the Markov property assumed in our previous work. In the first approach, we show that the excess entropy and the transfer entropy rate of an ergodic SSP can be obtained as the limits of permutation analogues of them for the N-th order approximation by hidden Markov models, respectively. In the second approach, we employ the modified permutation partition of the set of words which considers equalities of symbols in addition to permutations of words. We show that the excess entropy and the transfer entropy rate of an ergodic SSP are equal to their modified permutation analogues, respectively.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. C. Bandt, B. Pompe, Phys. Rev. Lett. 88, 174102 (2002)

    Article  ADS  Google Scholar 

  2. J.M. Amigó, Permutation Complexity in Dynamical Systems (Springer-Verlag Berlin Heidelberg, 2010)

  3. J.M. Amigó, M.B. Kennel, L. Kocarev, Physica D 210, 77 (2005)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  4. J.M. Amigó, M.B. Kennel, Physica D 231, 137 (2007)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  5. J.M. Amigó, Physica D 241, 789 (2012)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  6. C. Bandt, G. Keller, B. Pompe, Nonlinearity 15, 1595 (2002)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  7. K. Keller, M. Sinn, Physica D 239, 997 (2010)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  8. K. Keller, A.M. Unakafov, V.A. Unakafova, Physica D 241, 1477 (2012)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  9. T. Haruna, K. Nakajima, Physica D 240, 1370 (2011)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  10. J.P. Crutchfield, D.P. Feldman, Chaos 15, 25 (2003)

    Article  MathSciNet  ADS  Google Scholar 

  11. T. Haruna, K. Nakajima, Int. J. Comput. Ant. Sys. (in press)

  12. T. Schreiber, Phys. Rev. Lett. 85, 461 (2000)

    Article  ADS  Google Scholar 

  13. P.O. Amblard, O.J.J. Michel, J. Comput. Neurosci. 30, 7 (2011)

    Article  MathSciNet  Google Scholar 

  14. T. Haruna, K. Nakajima [arXiv:1112.2493]

  15. T. Haruna, K. Nakajima [arXiv:1204.1821]

  16. C. Bian, C. Qin, Q.D.Y. Ma, Q. Shen, Phys. Rev. E 85, 021906 (2012)

    Article  ADS  Google Scholar 

  17. M.C. Ruiz, A. Guillamón, A. Gabaldón, Entropy 14, 74 (2012)

    Article  Google Scholar 

  18. X. Sun, Y. Zou, V. Nikiforova, V. Kurths, D. Walther, BMC Biofinformatics 11, 607 (2010)

    Article  Google Scholar 

  19. R.B. Ash, Information Theory (Interscience Publishers, 1965)

  20. P. Billingsley, Convergence of Probability Measures, Second Edition (John Wiley & Sons, Inc., 1999)

  21. A. Kehagias, Ph.D. thesis, Brown University, 1992

  22. D.R. Upper, Ph.D. thesis, University of California, Berkeley, 1997

  23. P. Walters, An Introduction to Ergodic Theory (Springer-Verlag, New York, 1982)

  24. W. Löhr, Ph.D. thesis, Max-Planck-Institute for Mathematics in the Sciences, Leipzig, 2010

  25. T.M. Cover, J.A. Thomas, Elements of Information Theory (John Wiley & Sons, Inc., 1991)

  26. D.V. Arnold, Complex Systems 10, 143 (1996)

    MATH  Google Scholar 

  27. W. Bialek, I. Nemenman, N. Tishby, Neural Computation 13, 2409 (2001)

    Article  MATH  Google Scholar 

  28. D.P. Feldman, C.S. McTague, J.P. Crutchfield, Chaos 18, 043106 (2008)

    Article  MathSciNet  ADS  Google Scholar 

  29. P. Grassberger, Int. J. Theor. Phys. 25, 907 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  30. W. Li, Complex Systems 5, 381 (1991)

    MathSciNet  MATH  Google Scholar 

  31. R. Shaw, The Dripping Faucet as a Model Chaotic System (Aerial Press, Santa Cruz, California, 1984)

  32. R.E. Knop, J. Combi. Theor. A 15, 338 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  33. R.L. Graham, D.E. Knuth, O. Patashnik, Concrete Mathematics, Second Edition (Addison-Wesley Publishing Company, Inc., 1994)

  34. M. Staniek, K. Lehnertz, Phys. Rev. Lett. 100, 158101 (2008)

    Article  ADS  Google Scholar 

  35. D. Kugiumtzis, J. Nonlin. Sys. Appl. 3, 73 (2012)

    Google Scholar 

  36. R. Mullin, G.C. Rota, in Graph Theory and its Applications (Academic Press, 1970), p. 167

  37. S. Roman, G.C. Rota, Adv. Math. 27, 95 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  38. D. Senato, A. Venezia, Comput. Math. Appl. 41, 1109 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  39. A. Joyal, Adv. Math. 42, 1 (1981)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Earth & Planetary Sciences, Graduate School of Science, Kobe University, 1-1 Rokkodaicho, Nada, Kobe, 657-8501, Japan

    T. Haruna

  2. Department of Mechanical and Process Engineering, ETH Zurich, Leonhardstrasse 27, 8092, Zurich, Switzerland

    K. Nakajima

Authors
  1. T. Haruna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Haruna.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Haruna, T., Nakajima, K. Permutation approach to finite-alphabet stationary stochastic processes based on the duality between values and orderings. Eur. Phys. J. Spec. Top. 222, 383–399 (2013). https://doi.org/10.1140/epjst/e2013-01848-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1140/epjst/e2013-01848-5

Keywords

  • Entropy
  • Hide Markov Model
  • European Physical Journal Special Topic
  • Output Process
  • Symbolic Dynamics