The European Physical Journal B

, Volume 73, Issue 4, pp 605–615 | Cite as

Differentiating information transfer and causal effect

  • J. T. Lizier
  • M. Prokopenko
Interdisciplinary Physics


The concepts of information transfer and causal effect have received much recent attention, yet often the two are not appropriately distinguished and certain measures have been suggested to be suitable for both. We discuss two existing measures, transfer entropy and information flow, which can be used separately to quantify information transfer and causal information flow respectively. We apply these measures to cellular automata on a local scale in space and time, in order to explicitly contrast them and emphasize the differences between information transfer and causality. We also describe the manner in which the measures are complementary, including the conditions under which they in fact converge. We show that causal information flow is a primary tool to describe the causal structure of a system, while information transfer can then be used to describe the emergent computation on that causal structure.


Cellular Automaton Information Transfer Transfer Entropy Causal Information Conditional Mutual Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Phys. Rev. E 77, 026110 (2008) Google Scholar
  2. J. Pahle, A.K. Green, C.J. Dixon, U. Kummer, BMC Bioinformatics 9, 139 (2008) Google Scholar
  3. T.Q. Tung, T. Ryu, K.H. Lee, D. Lee, Inferring Gene Regulatory Networks from Microarray Time Series Data Using Transfer Entropy, in Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS ’07), Maribor, Slovenia, edited by P. Kokol, V. Podgorelec, D. Mičetič-Turk, M. Zorman, M. Verlič (IEEE, Los Alamitos, 2007), pp. 383–388 Google Scholar
  4. M. Lungarella, O. Sporns, PLoS Comput. Biol. 2, e144 (2006) Google Scholar
  5. X.S. Liang, Phys. Rev. E 78, 031113 (2008) Google Scholar
  6. N. Lüdtke, S. Panzeri, M. Brown, D.S. Broomhead, J. Knowles, M.A. Montemurro, D.B. Kell, J.R. Soc. Interface 5, 223 (2008) Google Scholar
  7. G. Auletta, G.F.R. Ellis, L. Jaeger, J.R. Soc. Interface 5, 1159 (2008) Google Scholar
  8. K. Hlaváčková-Schindler, M. Paluš, M. Vejmelka, J. Bhattacharya, Physics Reports 441, 1 (2007) Google Scholar
  9. T. Schreiber, Phys. Rev. Lett. 85, 461 (2000) Google Scholar
  10. N. Ay, D. Polani, Adv. Complex Syst. 11, 17 (2008) Google Scholar
  11. M. Lungarella, K. Ishiguro, Y. Kuniyoshi, N. Otsu, Int. J. Bifurcation Chaos 17, 903 (2007) Google Scholar
  12. K. Ishiguro, N. Otsu, M. Lungarella, Y. Kuniyoshi, Phys. Rev. E 77, 026216 (2008) Google Scholar
  13. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, A framework for the local information dynamics of distributed computation in complex systems (2008), e-print arXiv:0811.2690, Google Scholar
  14. H.B. Veatch, Aristotle: A contemporary appreciation (Indiana University Press, Bloomington, 1974) Google Scholar
  15. H. Sumioka, Y. Yoshikawa, M. Asada, Causality Detected by Transfer Entropy Leads Acquisition of Joint Attention, in Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL 2007), London (IEEE, 2007), pp. 264–269 Google Scholar
  16. M. Vejmelka, M. Palus, Phys. Rev. E 77, 026214 (2008) Google Scholar
  17. P.F. Verdes, Phys. Rev. E 72, 026222 (2005) Google Scholar
  18. G. Van Dijck, J. Van Vaerenbergh, M.M. Van Hulle, Information Theoretic Derivations for Causality Detection: Application to Human Gait, in Proceedings of the International Conference on Artificial Neural Networks (ICANN 2007), Porto, Portugal, edited by J.M.d. Sá, L.A. Alexandre, W. Duch, D. Mandic (Springer-Verlag, Berlin/Heidelberg, 2007), Lecture Notes in Computer Science, Vol. 4669, pp. 159–168 Google Scholar
  19. Y.C. Hung, C.K. Hu, Phys. Rev. Lett. 101, 244102 (2008) Google Scholar
  20. D.J. MacKay, Information Theory, Inference, and Learning Algorithms (Cambridge University Press, Cambridge, 2003) Google Scholar
  21. S. Wolfram, A New Kind of Science (Wolfram Media, Champaign, IL, USA, 2002) Google Scholar
  22. C.R. Shalizi, R. Haslinger, J.B. Rouquier, K.L. Klinkner, C. Moore, Phys. Rev. E 73, 036104 (2006) Google Scholar
  23. M. Mitchell, in Non-Standard Computation, edited by T. Gramss, S. Bornholdt, M. Gross, M. Mitchell, T. Pellizzari (VCH Verlagsgesellschaft, Weinheim, 1998), pp. 95–140 Google Scholar
  24. C.W.J. Granger, Econometrica 37, 424 (1969) Google Scholar
  25. T. Helvik, K. Lindgren, M.G. Nordahl, Comm. Math. Phys. 272, 53 (2007) Google Scholar
  26. J. Pearl, Causality: Models, Reasoning, and Inference (Cambridge University Press, Cambridge, 2000) Google Scholar
  27. L.R. Hope, K.B. Korb, Tech. Rep. 2005/176, Clayton School of Information Technology, Monash University (2005) Google Scholar
  28. A.S. Klyubin, D. Polani, C.L. Nehaniv, Tracking Information Flow through the Environment: Simple Cases of Stigmergy, in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALife IX), Boston, USA, edited by J. Pollack, M. Bedau, P. Husbands, T. Ikegami, R.A. Watson (MIT Press, Cambridge, MA, USA, 2004), pp. 563–568 Google Scholar
  29. J.E. Hanson, J.P. Crutchfield, J. Stat. Phys. 66, 1415 (1992) Google Scholar
  30. J.E. Hanson, J.P. Crutchfield, Physica D 103, 169 (1997) Google Scholar
  31. A. Wuensche, Complexity 4, 47 (1999) Google Scholar
  32. T. Helvik, K. Lindgren, M.G. Nordahl, Local information in one-dimensional cellular automata, in Proceedings of the International Conference on Cellular Automata for Research and Industry, Amsterdam, edited by P.M. Sloot, B. Chopard, A.G. Hoekstra (Springer, Berlin/Heidelberg, 2004), Lecture Notes in Computer Science, Vol. 3305, pp. 121–130 Google Scholar
  33. J.L. Mackie, in Causation, edited by E. Sosa, M. Tooley (Oxford University Press, New York, USA, 1993) Google Scholar
  34. M.R. DeWeese, M. Meister, Network: Computation in Neural Systems 10, 325 (1999) Google Scholar

Copyright information

© EDP Sciences, SIF, Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.CSIRO Information and Communications Technology CentreNorth RydeAustralia
  2. 2.School of Information Technologies, The University of SydneySydneyAustralia
  3. 3.Max Planck Institute for Mathematics in the SciencesLeipzigGermany

Personalised recommendations