Skip to main content

Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes

The European Physical Journal B volume 86, Article number: 230 (2013) Cite this article

Abstract

Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer entropy by counting the number of occurrences of orderings of values, not the values themselves. It has been suggested that the method of permutation is easy to implement, computationally low cost and robust to noise when applying to real world time series data. In this paper, we initiate a theoretical treatment of the corresponding rates. In particular, we consider the transfer entropy rate and its permutation analogue, the symbolic transfer entropy rate, and show that they are equal for any bivariate finite-alphabet stationary ergodic Markov process. This result is an illustration of the duality method introduced in [T. Haruna, K. Nakajima, Physica D 240, 1370 (2011)]. We also discuss the relationship among the transfer entropy rate, the time-delayed mutual information rate and their permutation analogues.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. T. Schreiber, Phys. Rev. Lett. 85, 461 (2000)

    Article  ADS  Google Scholar 

  2. M. Besserve, B. Schoelkopf, N.K. Logothetis, S. Panzeri, J. Comput. Neurosci. 29, 547 (2010)

    Article  Google Scholar 

  3. A. Buehlmann, G. Deco, PLoS Comput. Biol. 6, e1000934 (2010)

    Article  MathSciNet  Google Scholar 

  4. M. Garofalo, T. Nieus, P. Massobrio, S. Martinoia, PLoS ONE 4, e6482 (2009)

    Article  ADS  Google Scholar 

  5. C.J. Honey, R. Koetter, M. Breakspear, O. Sporns, Proc. Natl. Acad. Sci. USA 104, 10240 (2007)

    Article  ADS  Google Scholar 

  6. T. Katura, N. Tanaka, A. Obata, H. Sato, A. Maki, Neuroimage 31, 1592 (2006)

    Article  Google Scholar 

  7. N. Luedtke, N.K. Logothetis, S. Panzeri, Magn. Reson. Imag. 28, 1113 (2010)

    Article  Google Scholar 

  8. M. Lungarella, O. Sporns, PLoS Comput. Biol. 2, e144 (2006)

    Article  ADS  Google Scholar 

  9. S.A. Neymotin, K.M. Jacobs, A.A. Fenton, W.W. Lytton, J. Comput. Neurosci. 30, 69 (2011)

    Article  MathSciNet  Google Scholar 

  10. S. Sabesan, L.B. Good, K.S. Tsakalis, A. Spanias, D.M. Treiman, L.D. Iasemidis, IEEE Trans. Neural Syst. Rehabil. Eng. 17, 244 (2009)

    Article  Google Scholar 

  11. V.A. Vakorin, O.A. Krakovska, A.R. McIntosh, J. Neurosci. Methods 184, 152 (2009)

    Article  Google Scholar 

  12. M. Bauer, J.W. Cox, M.H. Caveness, J.J. Downs, N.F. Thornhill, IEEE Trans. Control Syst. Technol. 15, 12 (2007)

    Article  Google Scholar 

  13. H. Kamberaj, A. van der Vaart, Biophys. J. 97, 1747 (2009)

    Article  ADS  Google Scholar 

  14. R. Marschinski, H. Kantz, Eur. Phys. J. B 30, 275 (2002)

    Article  MathSciNet  ADS  Google Scholar 

  15. M. Materassi, A. Wernik, E. Yordanova, Nonlin. Processes Geophys. 14, 153 (2007)

    Article  ADS  Google Scholar 

  16. L.J. Moniz, E.G. Cooch, S.P. Ellner, J.D. Nichols, J.M. Nichols, Ecol. Modell. 208, 145 (2007)

    Article  Google Scholar 

  17. J.M. Nichols, M. Seaver, S.T. Trickey, M.D. Todd, C. Olson, L. Overbey, Phys. Rev. E 72, 046217 (2005)

    Article  ADS  Google Scholar 

  18. J. Pahle, A.K. Green, C.J. Dixon, U. Kummer, BMC Bioinf. 9, 139 (2008)

    Article  Google Scholar 

  19. T.Q. Tung, T. Ryu, K.H. Lee, D. Lee, in Proceedings of 20th IEEE International Symposium on Computer-Based Medical Systems, edited by P. Kokol, V. Podgorelec, D. MiceticTurk, M. Zorman, M. Verlic (2007), p. 383

  20. C. Bandt, B. Pompe, Phys. Rev. Lett. 88, 174102 (2002)

    Article  ADS  Google Scholar 

  21. C. Bandt, G. Keller, B. Pompe, Nonlinearity 15, 1595 (2002)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  22. C. Bandt, F. Shiha, J. Time Ser. Anal. 28, 646 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  23. J.M. Amigó, M.B. Kennel, Physica D 231, 137 (2007)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  24. M. Misiurewicz, Nonlinearity 16, 971 (2003)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  25. K. Keller, M. Sinn, Physica D 239, 997 (2010)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  26. K. Keller, A.M. Unakafov, V.A. Unakafova, Physica D 241, 1477 (2012)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  27. J.M. Amigó, Permutation Complexity in Dynamical Systems (Springer-Verlag, Berlin, Heidelberg, 2010)

  28. T.M. Cover, J.A. Thomas, Elements of Information Theory (John Wiley & Sons Inc., New York, 1991)

  29. J.M. Amigó, M.B. Kennel, L. Kocarev, Physica D 210, 77 (2005)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  30. J.M. Amigó, Physica D 241, 789 (2012)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  31. M. Staniek, K. Lehnertz, Phys. Rev. Lett. 100, 158101 (2008)

    Article  ADS  Google Scholar 

  32. A.M. Kowalski, M.T. Martin, A. Plastino, L. Zunino, Phys. Lett. A 374, 1819 (2010)

    Article  ADS  MATH  Google Scholar 

  33. M. Martini, T.A. Kranz, T. Wagner, K. Lehnertz, Phys. Rev. E 83, 011919 (2011)

    Article  MathSciNet  ADS  Google Scholar 

  34. A. Papana, D. Kugiumtzis, P.G. Larsson, Phys. Rev. E 83, 036207 (2011)

    Article  ADS  Google Scholar 

  35. D. Kugiumtzis, J. Nonlin. Sys. Appl. 3, 73 (2012)

    Google Scholar 

  36. P.O. Amblard, O.J.J. Michel, J. Comput. Neurosci. 30, 7 (2011)

    Article  MathSciNet  Google Scholar 

  37. T. Haruna, K. Nakajima, Physica D 240, 1370 (2011)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  38. J.P. Crutchfield, N.H. Packard, Physica D 7, 201 (1983)

    Article  MathSciNet  ADS  Google Scholar 

  39. J.P. Crutchfield, D.P. Feldman, Chaos 15, 25 (2003)

    Article  MathSciNet  ADS  Google Scholar 

  40. D.P. Feldman, C.S. McTague, J.P. Crutchfield, Chaos 18, 043106 (2008)

    Article  MathSciNet  ADS  Google Scholar 

  41. R. Shaw, The Dripping Faucet as a Model Chaotic System (Aerial Press, Santa Cruz, 1984)

  42. P. Grassberger, Int. J. Theor. Phys. 25, 907 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  43. W. Bialek, I. Nemenman, N. Tishby, Neural Comput. 13, 2409 (2001)

    Article  MATH  Google Scholar 

  44. W. Li, Complex Systems 5, 381 (1991)

    MathSciNet  MATH  Google Scholar 

  45. D.V. Arnold, Complex Systems 10, 143 (1996)

    MATH  Google Scholar 

  46. A. Katok, B. Hasselblatt, Introduction to the Modern Theory of Dynamical Systems (Cambridge University Press, Cambridge, 1995)

  47. A.M. Fraser, H.L. Swinney, Phys. Rev. A 33, 1134 (1986)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  48. K. Kaneko, Physica D 23, 436 (1986)

    Article  MathSciNet  ADS  Google Scholar 

  49. J.A. Vastano, H.L. Swinney, Phys. Rev. Lett. 60, 1773 (1988)

    Article  MathSciNet  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Earth & Planetary Sciences, Graduate School of Science, Kobe University, 1-1 Rokkodaicho, Nada, 657-8501, Kobe, Japan

    Taichi Haruna

  2. Department of Informatics, University of Zurich, Andreasstrasse 15, 8050, Zurich, Switzerland

    Kohei Nakajima

  3. Department of Mechanical and Process Engineering, ETH Zurich, Leonhardstrasse 27, 8092, Zurich, Switzerland

    Kohei Nakajima

Authors
  1. Taichi Haruna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kohei Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taichi Haruna.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Haruna, T., Nakajima, K. Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes. Eur. Phys. J. B 86, 230 (2013). https://doi.org/10.1140/epjb/e2013-30721-4

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1140/epjb/e2013-30721-4

Keywords

  • Statistical and Nonlinear Physics