Theory in Biosciences

, Volume 131, Issue 3, pp 193–203 | Cite as

Coherent information structure in complex computation

  • Joseph T. Lizier
  • Mikhail Prokopenko
  • Albert Y. Zomaya
Original Paper


We have recently presented a framework for the information dynamics of distributed computation that locally identifies the component operations of information storage, transfer, and modification. We have observed that while these component operations exist to some extent in all types of computation, complex computation is distinguished in having coherent structure in its local information dynamics profiles. In this article, we conjecture that coherent information structure is a defining feature of complex computation, particularly in biological systems or artificially evolved computation that solves human-understandable tasks. We present a methodology for studying coherent information structure, consisting of state-space diagrams of the local information dynamics and a measure of structure in these diagrams. The methodology identifies both clear and “hidden” coherent structure in complex computation, most notably reconciling conflicting interpretations of the complexity of the Elementary Cellular Automata rule 22.


Complex systems Coherent structure Information structure Emergence Information theory Information transfer Transfer entropy Information storage Cellular automata Self-organization 


  1. Badii R, Politi A (1997) Thermodynamics and complexity of cellular automata. Phys Rev Lett 78(3):444CrossRefGoogle Scholar
  2. Cavagna A, Cimarelli A, Giardina I, Parisi G, Santagati R, Stefanini F, Viale M (2010) Scale-free correlations in starling flocks. Proc Natl Acad Sci USA 107(26):11865–11870PubMedCrossRefGoogle Scholar
  3. Conway JH (1982) What is life? In: Berlekamp E, Conway JH, Guy R (eds) Winning ways for your mathematical plays, vol 2. Academic Press, New York, pp 927–962Google Scholar
  4. Couzin I, James R, Croft D, Krause J (2006) Social organization and information transfer in schooling fishes. In: Laland BCK, Krause J (eds) Fish cognition and behavior, fish and aquatic resources. Blackwell Publishing, Oxford, pp 166–185Google Scholar
  5. Crutchfield JP, Young K (1989) Inferring statistical complexity. Phys Rev Lett 63(2):105PubMedCrossRefGoogle Scholar
  6. Feldman D.P, Crutchfield J.P (2003) Structural information in two-dimensional patterns: entropy convergence and excess entropy. Phys Rev E 67(5):051104CrossRefGoogle Scholar
  7. Feldman DP, McTague CS, Crutchfield JP (2008) The organization of intrinsic computation: complexity–diagrams and the diversity of natural information processing. Chaos 18(4):043106PubMedCrossRefGoogle Scholar
  8. Fernández P, Solé RV (2006) The role of computation in complex regulatory networks. In: Koonin EV , Wolf YI , Karev GP (eds) Scale-free networks and genome biology. Landes Bioscience, Georgetown, pp 206–225Google Scholar
  9. Fernández P, Solé RV (2007) Neutral fitness landscapes in signalling networks. J R Soc Interface 4(12):41–47PubMedCrossRefGoogle Scholar
  10. Gong P, van Leeuwen C (2009) Distributed dynamical computation in neural circuits with propagating coherent activity patterns. PLoS Comput Biol 5(12):e1000611PubMedCrossRefGoogle Scholar
  11. Grassberger P (1986a) Long-range effects in an elementary cellular automaton. J Stat Phys 45(1–2):27–39CrossRefGoogle Scholar
  12. Grassberger P (1986b) Toward a quantitative theory of self-generated complexity. Int J Theor Phys 25(9):907–938CrossRefGoogle Scholar
  13. Gutowitz H, Domain C (1997) The topological skeleton of cellular automaton dynamics. Physica D 103(1–4):155–168Google Scholar
  14. Hanson JE, Crutchfield JP (1992) The attractor-basin portait of a cellular automaton. J Stat Phys 66:1415–1462CrossRefGoogle Scholar
  15. Helvik T, Lindgren K, Nordahl MG (2004) Local information in one-dimensional cellular automata. In: Sloot PM, Chopard B, Hoekstra AG (eds) Proceedings of the international conference on cellular automata for research and industry. Lecture notes in computer science, vol 3305. Springer, Berlin/Heidelberg, Amsterdam, pp 121–130Google Scholar
  16. Jung P, Wang J, Wackerbauer R, Showalter K (2000) Coherent structure analysis of spatiotemporal chaos. Phys Rev E 61(2):2095–2098CrossRefGoogle Scholar
  17. Kantz H, Schreiber T (1997) Nonlinear time series analysis. Cambridge University Press, CambridgeGoogle Scholar
  18. Lafusa A, Bossomaier T (2005) Hyperplane localisation of self-replicating and other complex cellular automata rules, vol 1. In: Proceedings of the 2005 IEEE congress on evolutionary computation. IEEE Press, Edinburgh, pp 844–849Google Scholar
  19. Langton CG (1990) Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42(1–3):12–37CrossRefGoogle Scholar
  20. Lizier JT (2010) The local information dynamics of distributed computation in complex systems. PhD thesis, The University of SydneyGoogle Scholar
  21. Lizier JT, Prokopenko M (2010) Differentiating information transfer and causal effect. Eur Phys J B 73(4):605–615CrossRefGoogle Scholar
  22. Lizier JT, Prokopenko M, Zomaya AY (2007) Detecting non-trivial computation in complex dynamics. In: Almeida e Costa F, Rocha LM, Costa E, Harvey I, Coutinho A (eds) Proceedings of the 9th European conference on artificial life (ECAL 2007), Lisbon. Lecture notes in artificial intelligence, vol 4648. Springer, Berlin/Heidelberg, pp 895–904Google Scholar
  23. Lizier JT, Prokopenko M, Zomaya AY (2008a) The information dynamics of phase transitions in random Boolean networks. In: Bullock S, Noble J, Watson R, Bedau MA (eds) Proceedings of the eleventh international conference on the simulation and synthesis of living systems (ALife XI), Winchester. MIT Press, Cambridge, pp 374–381Google Scholar
  24. Lizier JT, Prokopenko M, Zomaya AY (2008b) Local information transfer as a spatiotemporal filter for complex systems. Phys Rev E 77(2):026110CrossRefGoogle Scholar
  25. Lizier JT, Prokopenko M, Tanev I, Zomaya AY (2008c) Emergence of glider-like structures in a modular robotic system. In: Bullock S, Noble J, Watson R, Bedau MA (eds) Proceedings of the eleventh international conference on the simulation and synthesis of living systems (ALife XI), Winchester. MIT Press, Cambridge, pp 366–373 Google Scholar
  26. Lizier JT, Prokopenko M, Zomaya AY (2010a) Information modification and particle collisions in distributed computation. Chaos 20(3): 037109-13CrossRefGoogle Scholar
  27. Lizier JT, Prokopenko M, Zomaya AY (2010b) Local measures of information storage in complex distributed computation (in review)Google Scholar
  28. MacKay DJ (2003) Information theory, inference, and learning algorithms. Cambridge University Press, CambridgeGoogle Scholar
  29. McIntosh HV (1990) Linear cellular automata. Universidad Autónoma de Puebla, PueblaGoogle Scholar
  30. Mitchell M (1998) Computation in cellular automata: a selected review. In: Gramss T, Bornholdt S, Gross M, Mitchell M, Pellizzari T (eds) Non-standard computation. VCH Verlagsgesellschaft, Weinheim, pp 95–140Google Scholar
  31. Mitchell M, Crutchfield JP, Das R (1996) Evolving cellular automata with genetic algorithms: a review of recent work. In: Goodman ED, Punch W, Uskov V (eds) Proceedings of the first international conference on evolutionary computation and its applications. Russian Academy of Sciences, MoscowGoogle Scholar
  32. Mitchell M, Crutchfield JP, Hraber PT (1994) Evolving cellular automata to perform computations: mechanisms and impediments. Physica D 75:361–391CrossRefGoogle Scholar
  33. Oxford English Dictionary (2008) Accessed 8 May 2008
  34. Peak D, West JD, Messinger SM, Mott KA (2004) Evidence for complex, collective dynamics and emergent, distributed computation in plants. Proc Natl Acad Sci USA 101(4):918–922PubMedCrossRefGoogle Scholar
  35. Prokopenko M (2009) Guided self-organization. HFSP J 3(5):287–289PubMedCrossRefGoogle Scholar
  36. Prokopenko M, Gerasimov V, Tanev I (2006) Evolving spatiotemporal coordination in a modular robotic system. In: Nolfi S, Baldassarre G, Calabretta R, Hallam J, Marocco D, Meyer JA, Parisi D (eds) Proceedings of the ninth international conference on the simulation of adaptive behavior (SAB’06), Rome. Lecture notes in artificial intelligence, vol 4095. Springer Verlag, Berlin, pp 548–559Google Scholar
  37. Rouquier JB (2005) Cimula—a cellular automata analyser. Université de Lyon, Software
  38. Schreiber T (2000) Measuring information transfer. Phys Rev Lett 85(2):461–464PubMedCrossRefGoogle Scholar
  39. Shalizi CR, Haslinger R, Rouquier JB, Klinkner KL, Moore C (2006) Automatic filters for the detection of coherent structure in spatiotemporal systems. Phys Rev E 73(3):036104CrossRefGoogle Scholar
  40. Tononi G, Sporns O, Edelman G (1994) A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc Natl Acad Sci USA 91(11):5033–5037PubMedCrossRefGoogle Scholar
  41. Wolfram S (2002) A new kind of science. Wolfram Media, ChampaignGoogle Scholar
  42. Wuensche A (1999) Classifying cellular automata automatically: finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 4(3):47–66CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Joseph T. Lizier
    • 1
    • 2
    • 3
  • Mikhail Prokopenko
    • 1
  • Albert Y. Zomaya
    • 2
  1. 1.CSIRO Information and Communications Technology CentreEppingAustralia
  2. 2.School of Information TechnologiesThe University of SydneySydneyAustralia
  3. 3.Max Planck Institute for Mathematics in the SciencesLeipzigGermany

Personalised recommendations