Skip to main content

A Framework for the Local Information Dynamics of Distributed Computation in Complex Systems

  • Chapter

Part of the Emergence, Complexity and Computation book series (ECC,volume 9)

Abstract

The nature of distributed computation has long been a topic of interest in complex systems science, physics, artificial life and bioinformatics. In particular, emergent complex behavior has often been described from the perspective of computation within the system (Mitchell 1998b,a) and has been postulated to be associated with the capability to support universal computation (Langton 1990; Wolfram 1984c; Casti 1991).

Keywords

  • Cellular Automaton
  • Information Transfer
  • Cellular Automaton
  • Excess Entropy
  • Transfer Entropy

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Adamatzky, A. (ed.): Collision-Based Computing. Springer, Berlin (2002)

    MATH  Google Scholar 

  • Atick, J.J.: Could information theory provide an ecological theory of sensory processing? Network: Computation in Neural Systems 3(2), 213 (1992)

    CrossRef  MATH  Google Scholar 

  • Badii, R., Politi, A.: Thermodynamics and Complexity of Cellular Automata. Physical Review Letters 78(3), 444 (1997)

    CrossRef  Google Scholar 

  • Bialek, W., Nemenman, I., Tishby, N.: Complexity through nonextensivity. Physica A: Statistical Mechanics and its Applications 302(1-4), 89–99 (2001)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Boccara, N., Nasser, J., Roger, M.: Particlelike structures and their interactions in spatiotemporal patterns generated by one-dimensional deterministic cellular-Automaton rules. Physical Review A 44(2), 866–875 (1991)

    CrossRef  Google Scholar 

  • Boedecker, J., Obst, O., Lizier, J.T., Mayer, N.M., Asada, M.: Information processing in echo state networks at the edge of chaos. Theory in Biosciences 131(3), 205–213 (2012)

    CrossRef  Google Scholar 

  • Brown, J.A., Tuszynski, J.A.: A review of the ferroelectric model of microtubules. Ferroelectrics 220, 141–156 (1999)

    CrossRef  Google Scholar 

  • Casti, J.L.: Chaos, Gödel and truth. In: Casti, J.L., Karlqvist, A. (eds.) Beyond Belief: Randomness, Prediction and Explanation in Science, pp. 280–327. CRC Press, Boca Raton (1991)

    Google Scholar 

  • Ceguerra, R.V., Lizier, J.T., Zomaya, A.Y.: Information storage and transfer in the synchronization process in locally-connected networks. In: Proceedings of the 2011 IEEE Symposium on Artificial Life (ALIFE), pp. 54–61. IEEE (2011)

    Google Scholar 

  • Cliff, O.M., Lizier, J.T., Wang, X.R., Wang, P., Obst, O., Prokopenko, M.: Towards quantifying interaction networks in a football match. In: Proceedings of the RoboCup 2013 Symposium (to be published, 2013)

    Google Scholar 

  • Conway, J.H.: What is Life? In: Berlekamp, E., Conway, J.H., Guy, R. (eds.) Winning Ways for Your Mathematical Plays, vol. 2, ch. 25, pp. 927–962. Academic Press, New York (1982)

    Google Scholar 

  • Cook, M.: Universality in Elementary Cellular Automata. Complex Systems 15(1), 1–40 (2004)

    MATH  MathSciNet  Google Scholar 

  • Couzin, I.D., James, R., Croft, D.P., Krause, J.: Social Organization and Information Transfer in Schooling Fishes. In: Brown, C., Laland, K.N., Krause, J. (eds.) Fish Cognition and Behavior, Fish and Aquatic Resources, pp. 166–185. Blackwell Publishing (2006)

    Google Scholar 

  • Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley-Interscience, New York (1991)

    Google Scholar 

  • Crutchfield, J.P.: Personal communication (2009)

    Google Scholar 

  • Crutchfield, J.P., Ellison, C.J., Riechers, P.M.: Exact complexity: The spectral decomposition of intrinsic computation. arXiv:1309.3792 (2013)

    Google Scholar 

  • Crutchfield, J.P., Feldman, D.P.: Regularities Unseen, Randomness Observed: Levels of Entropy Convergence. Chaos 13(1), 25–54 (2003)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Crutchfield, J.P., Young, K.: Inferring statistical complexity. Physical Review Letters 63(2), 105–108 (1989)

    CrossRef  MathSciNet  Google Scholar 

  • Edmundson, D.E., Enns, R.H.: Fully 3-dimensional collisions of bistable light bullets. Optics Letters 18, 1609–1611 (1993)

    CrossRef  Google Scholar 

  • Eppstein, D.: Searching for spaceships. In: Nowakowski, R.J. (ed.) More Games of No Chance. MSRI Publications, vol. 42, pp. 433–453. Cambridge Univ. Press (2002)

    Google Scholar 

  • Fano, R.M.: Transmission of information: a statistical theory of communications. M.I.T. Press, Cambridge (1961)

    Google Scholar 

  • Feldman, D.P., McTague, C.S., Crutchfield, J.P.: The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing. Chaos 18(4), 43106 (2008)

    CrossRef  MathSciNet  Google Scholar 

  • Flecker, B., Alford, W., Beggs, J.M., Williams, P.L., Beer, R.D.: Partial information decomposition as a spatiotemporal filter. Chaos 21(3), 037104+ (2011)

    Google Scholar 

  • Goh, K.I., Barabási, A.L.: Burstiness and memory in complex systems. Europhysics Letters 81(4), 48002 (2008)

    CrossRef  MathSciNet  Google Scholar 

  • Grassberger, P.: New mechanism for deterministic diffusion. Physical Review A 28(6), 3666 (1983)

    CrossRef  Google Scholar 

  • Grassberger, P.: Long-range effects in an elementary cellular automaton. Journal of Statistical Physics 45(1-2), 27–39 (1986a)

    CrossRef  MathSciNet  Google Scholar 

  • Grassberger, P.: Toward a quantitative theory of self-generated complexity. International Journal of Theoretical Physics 25(9), 907–938 (1986b)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Grassberger, P.: Information content and predictability of lumped and distributed dynamical systems. Physica Scripta 40(3), 346 (1989)

    CrossRef  MathSciNet  Google Scholar 

  • Gray, L.: A Mathematician Looks at Wolfram’s New Kind of Science. Notices of the American Mathematical Society 50(2), 200–211 (2003)

    MATH  MathSciNet  Google Scholar 

  • Gutowitz, H., Domain, C.: The Topological Skeleton of Cellular Automaton Dynamics. Physica D 103(1-4), 155–168 (1997)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Hanson, J.E., Crutchfield, J.P.: The Attractor-Basin Portait of a Cellular Automaton. Journal of Statistical Physics 66, 1415–1462 (1992)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Hanson, J.E., Crutchfield, J.P.: Computational mechanics of cellular automata: An example. Physica D 103(1-4), 169–189 (1997)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Helvik, T., Lindgren, K., Nordahl, M.G.: Local information in one-dimensional cellular automata. In: Sloot, P.M.A., Chopard, B., Hoekstra, A.G. (eds.) ACRI 2004. LNCS, vol. 3305, pp. 121–130. Springer, Heidelberg (2004)

    CrossRef  Google Scholar 

  • Hordijk, W., Shalizi, C.R., Crutchfield, J.P.: Upper bound on the products of particle interactions in cellular automata. Physica D 154(3-4), 240–258 (2001)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Jakubowski, M.H., Steiglitz, K., Squier, R.: Information transfer between solitary waves in the saturable Schrödinger equation. Physical Review E 56(6), 7267 (1997)

    CrossRef  Google Scholar 

  • Jakubowski, M.H., Steiglitz, K., Squier, R.K.: Computing with solitons: A review and prospectus. Multiple-Valued Logic 6(5-6), 439–462 (2001)

    MATH  MathSciNet  Google Scholar 

  • Kinouchi, O., Copelli, M.: Optimal dynamical range of excitable networks at criticality. Nature Physics 2(5), 348–351 (2006)

    CrossRef  Google Scholar 

  • Klyubin, A.S., Polani, D., Nehaniv, C.L.: Tracking Information Flow through the Environment: Simple Cases of Stigmergy. In: Pollack, J., Bedau, M., Husbands, P., Ikegami, T., Watson, R.A. (eds.) Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALife IX), Boston, USA, pp. 563–568. MIT Press, Cambridge (2004)

    Google Scholar 

  • Klyubin, A.S., Polani, D., Nehaniv, C.L.: All Else Being Equal Be Empowered. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS (LNAI), vol. 3630, pp. 744–753. Springer, Heidelberg (2005)

    CrossRef  Google Scholar 

  • Lafusa, A., Bossomaier, T.: Hyperplane Localisation of Self-Replicating and Other Complex Cellular Automata Rules. In: Proceedings of the the 2005 IEEE Congress on Evolutionary Computation, Edinburgh, vol. 1, pp. 844–849. IEEE Press (2005)

    Google Scholar 

  • Langton, C.G.: Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42(1-3), 12–37 (1990)

    CrossRef  MathSciNet  Google Scholar 

  • Lindgren, K., Nordahl, M.G.: Complexity Measures and Cellular Automata. Complex Systems 2(4), 409–440 (1988)

    MATH  MathSciNet  Google Scholar 

  • Lindgren, K., Nordahl, M.G.: Universal computation in simple one-dimensional cellular automata. Complex Systems 4, 299–318 (1990)

    MATH  MathSciNet  Google Scholar 

  • Lindner, M., Vicente, R., Priesemann, V., Wibral, M.: TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neuroscience 12(1), 119+ (2011)

    Google Scholar 

  • Lizier, J.T.: JIDT: An information-theoretic toolkit for studying the dynamics of complex systems (2012), https://code.google.com/p/information-dynamics-toolkit/

  • Lizier, J.T.: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses. Springer, Heidelberg (2013)

    CrossRef  MATH  Google Scholar 

  • Lizier, J.T., Atay, F.M., Jost, J.: Information storage, loop motifs, and clustered structure in complex networks. Physical Review E 86(2), 026110+ (2012a)

    Google Scholar 

  • Lizier, J.T., Flecker, B., Williams, P.L.: Towards a synergy-based approach to measuring information modification. In: Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), pp. 43–51. IEEE (2013)

    Google Scholar 

  • Lizier, J.T., Heinzle, J., Horstmann, A., Haynes, J.-D., Prokopenko, M.: Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. Journal of Computational Neuroscience 30(1), 85–107 (2011a)

    CrossRef  MathSciNet  Google Scholar 

  • Lizier, J.T., Mahoney, J.R.: Moving frames of reference, relativity and invariance in transfer entropy and information dynamics. Entropy 15(1), 177–197 (2013)

    CrossRef  MathSciNet  Google Scholar 

  • Lizier, J.T., Pritam, S., Prokopenko, M.: Information dynamics in small-world Boolean networks. Artificial Life 17(4), 293–314 (2011b)

    CrossRef  Google Scholar 

  • Lizier, J.T., Prokopenko, M.: Differentiating information transfer and causal effect. European Physical Journal B 73(4), 605–615 (2010)

    CrossRef  Google Scholar 

  • Lizier, J.T., Prokopenko, M., Tanev, I., Zomaya, A.Y.: Emergence of Glider-like Structures in a Modular Robotic System. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, pp. 366–373. MIT Press, Cambridge (2008a)

    Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Detecting Non-trivial Computation in Complex Dynamics. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 895–904. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: The information dynamics of phase transitions in random Boolean networks. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, pp. 374–381. MIT Press, Cambridge (2008b)

    Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local information transfer as a spatiotemporal filter for complex systems. Physical Review E 77(2), 026110+ (2008c)

    Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Information modification and particle collisions in distributed computation. Chaos 20(3), 037109+ (2010)

    Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Coherent information structure in complex computation. Theory in Biosciences 131(3), 193–203 (2012b)

    CrossRef  Google Scholar 

  • Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local measures of information storage in complex distributed computation. Information Sciences 208, 39–54 (2012c)

    CrossRef  Google Scholar 

  • Lungarella, M., Sporns, O.: Mapping information flow in sensorimotor networks. PLoS Computational Biology 2(10), e144+ (2006)

    Google Scholar 

  • MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  • Marinazzo, D., Wu, G., Pellicoro, M., Angelini, L., Stramaglia, S.: Information flow in networks and the law of diminishing marginal returns: evidence from modeling and human electroencephalographic recordings. PloS ONE 7(9), e45026+ (2012)

    Google Scholar 

  • Martinez, G.J., Adamatzky, A., McIntosh, H.V.: Phenomenology of glider collisions in cellular automaton Rule 54 and associated logical gates. Chaos, Solitons and Fractals 28(1), 100–111 (2006)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • McIntosh, H.V.: Linear Cellular Automata. Universidad Autónoma de Puebla, Puebla, Mexico (1990)

    Google Scholar 

  • Mitchell, M.: A Complex-Systems Perspective on the “Computation vs. Dynamics” Debate in Cognitive Science. In: Gernsbacher, M.A., Derry, S.J. (eds.) Proceedings of the 20th Annual Conference of the Cognitive Science Society (Cogsci 1998), Madison, Wisconsin, pp. 710–715 (1998a)

    Google Scholar 

  • Mitchell, M.: Computation in Cellular Automata: A Selected Review. In: Gramss, T., Bornholdt, S., Gross, M., Mitchell, M., Pellizzari, T. (eds.) Non-Standard Computation, pp. 95–140. VCH Verlagsgesellschaft, Weinheim (1998b)

    Google Scholar 

  • Mitchell, M., Crutchfield, J.P., Das, R.: Evolving Cellular Automata with Genetic Algorithms: A Review of Recent Work. In: Goodman, E.D., Punch, W., Uskov, V. (eds.) Proceedings of the First International Conference on Evolutionary Computation and Its Applications, Moscow, Russia, Russian Academy of Sciences (1996)

    Google Scholar 

  • Mitchell, M., Crutchfield, J.P., Hraber, P.T.: Evolving Cellular Automata to Perform Computations: Mechanisms and Impediments. Physica D 75, 361–391 (1994)

    CrossRef  MATH  Google Scholar 

  • Morgado, R., Cieśla, M., Longa, L., Oliveira, F.A.: Synchronization in the presence of memory. Europhysics Letters 79(1), 10002 (2007)

    CrossRef  MathSciNet  Google Scholar 

  • Obst, O., Boedecker, J., Schmidt, B., Asada, M.: On active information storage in input-driven systems. arXiv:1303.5526 (2013)

    Google Scholar 

  • Oxford English Dictionary (2008), http://www.oed.com/ (accessed August 5, 2008)

  • Pahle, J., Green, A.K., Dixon, C.J., Kummer, U.: Information transfer in signaling pathways: a study using coupled simulated and experimental data. BMC Bioinformatics 9, 139 (2008)

    CrossRef  Google Scholar 

  • Prokopenko, M., Boschietti, F., Ryan, A.J.: An Information-Theoretic Primer on Complexity, Self-Organization, and Emergence. Complexity 15(1), 11–28 (2009)

    CrossRef  MathSciNet  Google Scholar 

  • Prokopenko, M., Gerasimov, V., Tanev, I.: Evolving Spatiotemporal Coordination in a Modular Robotic System. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 558–569. Springer, Heidelberg (2006)

    CrossRef  Google Scholar 

  • Prokopenko, M., Lizier, J.T., Obst, O., Wang, X.R.: Relating Fisher information to order parameters. Physical Review E 84, 041116+ (2011)

    Google Scholar 

  • Prokopenko, M., Lizier, J.T., Price, D.C.: On thermodynamic interpretation of transfer entropy. Entropy 15(2), 524–543 (2013)

    CrossRef  MathSciNet  Google Scholar 

  • Sánchez-Montañés, M.A., Corbacho, F.J.: Towards a New Information Processing Measure for Neural Computation. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 637–642. Springer, Heidelberg (2002)

    CrossRef  Google Scholar 

  • Schreiber, T.: Measuring Information Transfer. Physical Review Letters 85(2), 461–464 (2000)

    CrossRef  Google Scholar 

  • Shalizi, C.R.: Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata. PhD thesis, University of Wisconsin-Madison (2001)

    Google Scholar 

  • Shalizi, C.R., Crutchfield, J.P.: Computational mechanics: Pattern and Prediction, Structure and Simplicity. Journal of Statistical Physics 104, 817–879 (2001)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Shalizi, C.R., Haslinger, R., Rouquier, J.-B., Klinkner, K.L., Moore, C.: Automatic filters for the detection of coherent structure in spatiotemporal systems. Physical Review E 73(3), 036104 (2006)

    Google Scholar 

  • Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27 (1948)

    Google Scholar 

  • Takens, F.: Detecting strange attractors in turbulence. In: Rand, D., Young, L.-S. (eds.) Dynamical Systems and Turbulence, Warwick 1980. Lecture Notes in Mathematics, vol. 21, pp. 366–381. Springer, Heidelberg (1981)

    CrossRef  Google Scholar 

  • Von Neumann, J.: Theory of self-reproducing automata. University of Illinois Press, Urbana (1966)

    Google Scholar 

  • Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.: Quantifying and Tracing Information Cascades in Swarms. PLoS ONE 7(7), e40084+ (2012)

    Google Scholar 

  • Wibral, M., Pampu, N., Priesemann, V., Siebenhühner, F., Seiwert, H., Lindner, M., Lizier, J.T., Vicente, R.: Measuring Information-Transfer delays. PLoS ONE 8(2), e55809+ (2013)

    Google Scholar 

  • Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. Progress in Biophysics and Molecular Biology 105(1-2), 80–97 (2011)

    CrossRef  Google Scholar 

  • Williams, P.L., Beer, R.D.: Nonnegative Decomposition of Multivariate Information. arXiv:1004.2515 (2010)

    Google Scholar 

  • Wolfram, S.: Cellular automata as models of complexity. Nature 311(5985), 419–424 (1984a)

    CrossRef  Google Scholar 

  • Wolfram, S.: Computation theory of cellular automata. Communications in Mathematical Physics 96(1), 15–57 (1984b)

    CrossRef  MATH  MathSciNet  Google Scholar 

  • Wolfram, S.: Universality and complexity in cellular automata. Physica D 10(1-2), 1–35 (1984c)

    CrossRef  MathSciNet  Google Scholar 

  • Wolfram, S.: A New Kind of Science. Wolfram Media, Champaign (2002)

    MATH  Google Scholar 

  • Wuensche, A.: Classifying cellular automata automatically: Finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 4(3), 47–66 (1999)

    CrossRef  MathSciNet  Google Scholar 

  • Yamada, T., Aihara, K.: Spatio-temporal complex dynamics and computation in chaotic neural networks. In: Proceedings of the IEEE Symposium on Emerging Technologies and Factory Automation (ETFA 1994), Tokyo, pp. 239–244. IEEE (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Lizier, J.T., Prokopenko, M., Zomaya, A.Y. (2014). A Framework for the Local Information Dynamics of Distributed Computation in Complex Systems. In: Prokopenko, M. (eds) Guided Self-Organization: Inception. Emergence, Complexity and Computation, vol 9. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53734-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-53734-9_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-53733-2

  • Online ISBN: 978-3-642-53734-9

  • eBook Packages: EngineeringEngineering (R0)