Skip to main content

Differentiating information transfer and causal effect

Abstract

The concepts of information transfer and causal effect have received much recent attention, yet often the two are not appropriately distinguished and certain measures have been suggested to be suitable for both. We discuss two existing measures, transfer entropy and information flow, which can be used separately to quantify information transfer and causal information flow respectively. We apply these measures to cellular automata on a local scale in space and time, in order to explicitly contrast them and emphasize the differences between information transfer and causality. We also describe the manner in which the measures are complementary, including the conditions under which they in fact converge. We show that causal information flow is a primary tool to describe the causal structure of a system, while information transfer can then be used to describe the emergent computation on that causal structure.

This is a preview of subscription content, access via your institution.

References

  1. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Phys. Rev. E 77, 026110 (2008)

  2. J. Pahle, A.K. Green, C.J. Dixon, U. Kummer, BMC Bioinformatics 9, 139 (2008)

  3. T.Q. Tung, T. Ryu, K.H. Lee, D. Lee, Inferring Gene Regulatory Networks from Microarray Time Series Data Using Transfer Entropy, in Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS ’07), Maribor, Slovenia, edited by P. Kokol, V. Podgorelec, D. Mičetič-Turk, M. Zorman, M. Verlič (IEEE, Los Alamitos, 2007), pp. 383–388

  4. M. Lungarella, O. Sporns, PLoS Comput. Biol. 2, e144 (2006)

  5. X.S. Liang, Phys. Rev. E 78, 031113 (2008)

    Google Scholar 

  6. N. Lüdtke, S. Panzeri, M. Brown, D.S. Broomhead, J. Knowles, M.A. Montemurro, D.B. Kell, J.R. Soc. Interface 5, 223 (2008)

    Google Scholar 

  7. G. Auletta, G.F.R. Ellis, L. Jaeger, J.R. Soc. Interface 5, 1159 (2008)

  8. K. Hlaváčková-Schindler, M. Paluš, M. Vejmelka, J. Bhattacharya, Physics Reports 441, 1 (2007)

    Google Scholar 

  9. T. Schreiber, Phys. Rev. Lett. 85, 461 (2000)

    Google Scholar 

  10. N. Ay, D. Polani, Adv. Complex Syst. 11, 17 (2008)

  11. M. Lungarella, K. Ishiguro, Y. Kuniyoshi, N. Otsu, Int. J. Bifurcation Chaos 17, 903 (2007)

    Google Scholar 

  12. K. Ishiguro, N. Otsu, M. Lungarella, Y. Kuniyoshi, Phys. Rev. E 77, 026216 (2008)

    Google Scholar 

  13. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, A framework for the local information dynamics of distributed computation in complex systems (2008), e-print arXiv:0811.2690, http://arxiv.org/abs/0811.2690

  14. H.B. Veatch, Aristotle: A contemporary appreciation (Indiana University Press, Bloomington, 1974)

  15. H. Sumioka, Y. Yoshikawa, M. Asada, Causality Detected by Transfer Entropy Leads Acquisition of Joint Attention, in Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL 2007), London (IEEE, 2007), pp. 264–269

  16. M. Vejmelka, M. Palus, Phys. Rev. E 77, 026214 (2008)

    Google Scholar 

  17. P.F. Verdes, Phys. Rev. E 72, 026222 (2005)

    Google Scholar 

  18. G. Van Dijck, J. Van Vaerenbergh, M.M. Van Hulle, Information Theoretic Derivations for Causality Detection: Application to Human Gait, in Proceedings of the International Conference on Artificial Neural Networks (ICANN 2007), Porto, Portugal, edited by J.M.d. Sá, L.A. Alexandre, W. Duch, D. Mandic (Springer-Verlag, Berlin/Heidelberg, 2007), Lecture Notes in Computer Science, Vol. 4669, pp. 159–168

  19. Y.C. Hung, C.K. Hu, Phys. Rev. Lett. 101, 244102 (2008)

    Google Scholar 

  20. D.J. MacKay, Information Theory, Inference, and Learning Algorithms (Cambridge University Press, Cambridge, 2003)

  21. S. Wolfram, A New Kind of Science (Wolfram Media, Champaign, IL, USA, 2002)

  22. C.R. Shalizi, R. Haslinger, J.B. Rouquier, K.L. Klinkner, C. Moore, Phys. Rev. E 73, 036104 (2006)

    Google Scholar 

  23. M. Mitchell, in Non-Standard Computation, edited by T. Gramss, S. Bornholdt, M. Gross, M. Mitchell, T. Pellizzari (VCH Verlagsgesellschaft, Weinheim, 1998), pp. 95–140

  24. C.W.J. Granger, Econometrica 37, 424 (1969)

  25. T. Helvik, K. Lindgren, M.G. Nordahl, Comm. Math. Phys. 272, 53 (2007)

  26. J. Pearl, Causality: Models, Reasoning, and Inference (Cambridge University Press, Cambridge, 2000)

  27. L.R. Hope, K.B. Korb, Tech. Rep. 2005/176, Clayton School of Information Technology, Monash University (2005)

  28. A.S. Klyubin, D. Polani, C.L. Nehaniv, Tracking Information Flow through the Environment: Simple Cases of Stigmergy, in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALife IX), Boston, USA, edited by J. Pollack, M. Bedau, P. Husbands, T. Ikegami, R.A. Watson (MIT Press, Cambridge, MA, USA, 2004), pp. 563–568

  29. J.E. Hanson, J.P. Crutchfield, J. Stat. Phys. 66, 1415 (1992)

    Google Scholar 

  30. J.E. Hanson, J.P. Crutchfield, Physica D 103, 169 (1997)

    Google Scholar 

  31. A. Wuensche, Complexity 4, 47 (1999)

  32. T. Helvik, K. Lindgren, M.G. Nordahl, Local information in one-dimensional cellular automata, in Proceedings of the International Conference on Cellular Automata for Research and Industry, Amsterdam, edited by P.M. Sloot, B. Chopard, A.G. Hoekstra (Springer, Berlin/Heidelberg, 2004), Lecture Notes in Computer Science, Vol. 3305, pp. 121–130

  33. J.L. Mackie, in Causation, edited by E. Sosa, M. Tooley (Oxford University Press, New York, USA, 1993)

  34. M.R. DeWeese, M. Meister, Network: Computation in Neural Systems 10, 325 (1999)

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to J. T. Lizier.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lizier, J., Prokopenko, M. Differentiating information transfer and causal effect. Eur. Phys. J. B 73, 605–615 (2010). https://doi.org/10.1140/epjb/e2010-00034-5

Download citation

Keywords

  • Cellular Automaton
  • Information Transfer
  • Transfer Entropy
  • Causal Information
  • Conditional Mutual Information