Abstract
The concepts of information transfer and causal effect have received much recent attention, yet often the two are not appropriately distinguished and certain measures have been suggested to be suitable for both. We discuss two existing measures, transfer entropy and information flow, which can be used separately to quantify information transfer and causal information flow respectively. We apply these measures to cellular automata on a local scale in space and time, in order to explicitly contrast them and emphasize the differences between information transfer and causality. We also describe the manner in which the measures are complementary, including the conditions under which they in fact converge. We show that causal information flow is a primary tool to describe the causal structure of a system, while information transfer can then be used to describe the emergent computation on that causal structure.
This is a preview of subscription content, access via your institution.
References
J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Phys. Rev. E 77, 026110 (2008)
J. Pahle, A.K. Green, C.J. Dixon, U. Kummer, BMC Bioinformatics 9, 139 (2008)
T.Q. Tung, T. Ryu, K.H. Lee, D. Lee, Inferring Gene Regulatory Networks from Microarray Time Series Data Using Transfer Entropy, in Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS ’07), Maribor, Slovenia, edited by P. Kokol, V. Podgorelec, D. Mičetič-Turk, M. Zorman, M. Verlič (IEEE, Los Alamitos, 2007), pp. 383–388
M. Lungarella, O. Sporns, PLoS Comput. Biol. 2, e144 (2006)
X.S. Liang, Phys. Rev. E 78, 031113 (2008)
N. Lüdtke, S. Panzeri, M. Brown, D.S. Broomhead, J. Knowles, M.A. Montemurro, D.B. Kell, J.R. Soc. Interface 5, 223 (2008)
G. Auletta, G.F.R. Ellis, L. Jaeger, J.R. Soc. Interface 5, 1159 (2008)
K. Hlaváčková-Schindler, M. Paluš, M. Vejmelka, J. Bhattacharya, Physics Reports 441, 1 (2007)
T. Schreiber, Phys. Rev. Lett. 85, 461 (2000)
N. Ay, D. Polani, Adv. Complex Syst. 11, 17 (2008)
M. Lungarella, K. Ishiguro, Y. Kuniyoshi, N. Otsu, Int. J. Bifurcation Chaos 17, 903 (2007)
K. Ishiguro, N. Otsu, M. Lungarella, Y. Kuniyoshi, Phys. Rev. E 77, 026216 (2008)
J.T. Lizier, M. Prokopenko, A.Y. Zomaya, A framework for the local information dynamics of distributed computation in complex systems (2008), e-print arXiv:0811.2690, http://arxiv.org/abs/0811.2690
H.B. Veatch, Aristotle: A contemporary appreciation (Indiana University Press, Bloomington, 1974)
H. Sumioka, Y. Yoshikawa, M. Asada, Causality Detected by Transfer Entropy Leads Acquisition of Joint Attention, in Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL 2007), London (IEEE, 2007), pp. 264–269
M. Vejmelka, M. Palus, Phys. Rev. E 77, 026214 (2008)
P.F. Verdes, Phys. Rev. E 72, 026222 (2005)
G. Van Dijck, J. Van Vaerenbergh, M.M. Van Hulle, Information Theoretic Derivations for Causality Detection: Application to Human Gait, in Proceedings of the International Conference on Artificial Neural Networks (ICANN 2007), Porto, Portugal, edited by J.M.d. Sá, L.A. Alexandre, W. Duch, D. Mandic (Springer-Verlag, Berlin/Heidelberg, 2007), Lecture Notes in Computer Science, Vol. 4669, pp. 159–168
Y.C. Hung, C.K. Hu, Phys. Rev. Lett. 101, 244102 (2008)
D.J. MacKay, Information Theory, Inference, and Learning Algorithms (Cambridge University Press, Cambridge, 2003)
S. Wolfram, A New Kind of Science (Wolfram Media, Champaign, IL, USA, 2002)
C.R. Shalizi, R. Haslinger, J.B. Rouquier, K.L. Klinkner, C. Moore, Phys. Rev. E 73, 036104 (2006)
M. Mitchell, in Non-Standard Computation, edited by T. Gramss, S. Bornholdt, M. Gross, M. Mitchell, T. Pellizzari (VCH Verlagsgesellschaft, Weinheim, 1998), pp. 95–140
C.W.J. Granger, Econometrica 37, 424 (1969)
T. Helvik, K. Lindgren, M.G. Nordahl, Comm. Math. Phys. 272, 53 (2007)
J. Pearl, Causality: Models, Reasoning, and Inference (Cambridge University Press, Cambridge, 2000)
L.R. Hope, K.B. Korb, Tech. Rep. 2005/176, Clayton School of Information Technology, Monash University (2005)
A.S. Klyubin, D. Polani, C.L. Nehaniv, Tracking Information Flow through the Environment: Simple Cases of Stigmergy, in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALife IX), Boston, USA, edited by J. Pollack, M. Bedau, P. Husbands, T. Ikegami, R.A. Watson (MIT Press, Cambridge, MA, USA, 2004), pp. 563–568
J.E. Hanson, J.P. Crutchfield, J. Stat. Phys. 66, 1415 (1992)
J.E. Hanson, J.P. Crutchfield, Physica D 103, 169 (1997)
A. Wuensche, Complexity 4, 47 (1999)
T. Helvik, K. Lindgren, M.G. Nordahl, Local information in one-dimensional cellular automata, in Proceedings of the International Conference on Cellular Automata for Research and Industry, Amsterdam, edited by P.M. Sloot, B. Chopard, A.G. Hoekstra (Springer, Berlin/Heidelberg, 2004), Lecture Notes in Computer Science, Vol. 3305, pp. 121–130
J.L. Mackie, in Causation, edited by E. Sosa, M. Tooley (Oxford University Press, New York, USA, 1993)
M.R. DeWeese, M. Meister, Network: Computation in Neural Systems 10, 325 (1999)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lizier, J., Prokopenko, M. Differentiating information transfer and causal effect. Eur. Phys. J. B 73, 605–615 (2010). https://doi.org/10.1140/epjb/e2010-00034-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1140/epjb/e2010-00034-5