Abstract
The nature of distributed computation has long been a topic of interest in complex systems science, physics, artificial life and bioinformatics. In particular, emergent complex behavior has often been described from the perspective of computation within the system (Mitchell 1998b,a) and has been postulated to be associated with the capability to support universal computation (Langton 1990; Wolfram 1984c; Casti 1991).
Keywords
- Cellular Automaton
- Information Transfer
- Cellular Automaton
- Excess Entropy
- Transfer Entropy
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Adamatzky, A. (ed.): Collision-Based Computing. Springer, Berlin (2002)
Atick, J.J.: Could information theory provide an ecological theory of sensory processing? Network: Computation in Neural Systems 3(2), 213 (1992)
Badii, R., Politi, A.: Thermodynamics and Complexity of Cellular Automata. Physical Review Letters 78(3), 444 (1997)
Bialek, W., Nemenman, I., Tishby, N.: Complexity through nonextensivity. Physica A: Statistical Mechanics and its Applications 302(1-4), 89–99 (2001)
Boccara, N., Nasser, J., Roger, M.: Particlelike structures and their interactions in spatiotemporal patterns generated by one-dimensional deterministic cellular-Automaton rules. Physical Review A 44(2), 866–875 (1991)
Boedecker, J., Obst, O., Lizier, J.T., Mayer, N.M., Asada, M.: Information processing in echo state networks at the edge of chaos. Theory in Biosciences 131(3), 205–213 (2012)
Brown, J.A., Tuszynski, J.A.: A review of the ferroelectric model of microtubules. Ferroelectrics 220, 141–156 (1999)
Casti, J.L.: Chaos, Gödel and truth. In: Casti, J.L., Karlqvist, A. (eds.) Beyond Belief: Randomness, Prediction and Explanation in Science, pp. 280–327. CRC Press, Boca Raton (1991)
Ceguerra, R.V., Lizier, J.T., Zomaya, A.Y.: Information storage and transfer in the synchronization process in locally-connected networks. In: Proceedings of the 2011 IEEE Symposium on Artificial Life (ALIFE), pp. 54–61. IEEE (2011)
Cliff, O.M., Lizier, J.T., Wang, X.R., Wang, P., Obst, O., Prokopenko, M.: Towards quantifying interaction networks in a football match. In: Proceedings of the RoboCup 2013 Symposium (to be published, 2013)
Conway, J.H.: What is Life? In: Berlekamp, E., Conway, J.H., Guy, R. (eds.) Winning Ways for Your Mathematical Plays, vol. 2, ch. 25, pp. 927–962. Academic Press, New York (1982)
Cook, M.: Universality in Elementary Cellular Automata. Complex Systems 15(1), 1–40 (2004)
Couzin, I.D., James, R., Croft, D.P., Krause, J.: Social Organization and Information Transfer in Schooling Fishes. In: Brown, C., Laland, K.N., Krause, J. (eds.) Fish Cognition and Behavior, Fish and Aquatic Resources, pp. 166–185. Blackwell Publishing (2006)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley-Interscience, New York (1991)
Crutchfield, J.P.: Personal communication (2009)
Crutchfield, J.P., Ellison, C.J., Riechers, P.M.: Exact complexity: The spectral decomposition of intrinsic computation. arXiv:1309.3792 (2013)
Crutchfield, J.P., Feldman, D.P.: Regularities Unseen, Randomness Observed: Levels of Entropy Convergence. Chaos 13(1), 25–54 (2003)
Crutchfield, J.P., Young, K.: Inferring statistical complexity. Physical Review Letters 63(2), 105–108 (1989)
Edmundson, D.E., Enns, R.H.: Fully 3-dimensional collisions of bistable light bullets. Optics Letters 18, 1609–1611 (1993)
Eppstein, D.: Searching for spaceships. In: Nowakowski, R.J. (ed.) More Games of No Chance. MSRI Publications, vol. 42, pp. 433–453. Cambridge Univ. Press (2002)
Fano, R.M.: Transmission of information: a statistical theory of communications. M.I.T. Press, Cambridge (1961)
Feldman, D.P., McTague, C.S., Crutchfield, J.P.: The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing. Chaos 18(4), 43106 (2008)
Flecker, B., Alford, W., Beggs, J.M., Williams, P.L., Beer, R.D.: Partial information decomposition as a spatiotemporal filter. Chaos 21(3), 037104+ (2011)
Goh, K.I., Barabási, A.L.: Burstiness and memory in complex systems. Europhysics Letters 81(4), 48002 (2008)
Grassberger, P.: New mechanism for deterministic diffusion. Physical Review A 28(6), 3666 (1983)
Grassberger, P.: Long-range effects in an elementary cellular automaton. Journal of Statistical Physics 45(1-2), 27–39 (1986a)
Grassberger, P.: Toward a quantitative theory of self-generated complexity. International Journal of Theoretical Physics 25(9), 907–938 (1986b)
Grassberger, P.: Information content and predictability of lumped and distributed dynamical systems. Physica Scripta 40(3), 346 (1989)
Gray, L.: A Mathematician Looks at Wolfram’s New Kind of Science. Notices of the American Mathematical Society 50(2), 200–211 (2003)
Gutowitz, H., Domain, C.: The Topological Skeleton of Cellular Automaton Dynamics. Physica D 103(1-4), 155–168 (1997)
Hanson, J.E., Crutchfield, J.P.: The Attractor-Basin Portait of a Cellular Automaton. Journal of Statistical Physics 66, 1415–1462 (1992)
Hanson, J.E., Crutchfield, J.P.: Computational mechanics of cellular automata: An example. Physica D 103(1-4), 169–189 (1997)
Helvik, T., Lindgren, K., Nordahl, M.G.: Local information in one-dimensional cellular automata. In: Sloot, P.M.A., Chopard, B., Hoekstra, A.G. (eds.) ACRI 2004. LNCS, vol. 3305, pp. 121–130. Springer, Heidelberg (2004)
Hordijk, W., Shalizi, C.R., Crutchfield, J.P.: Upper bound on the products of particle interactions in cellular automata. Physica D 154(3-4), 240–258 (2001)
Jakubowski, M.H., Steiglitz, K., Squier, R.: Information transfer between solitary waves in the saturable Schrödinger equation. Physical Review E 56(6), 7267 (1997)
Jakubowski, M.H., Steiglitz, K., Squier, R.K.: Computing with solitons: A review and prospectus. Multiple-Valued Logic 6(5-6), 439–462 (2001)
Kinouchi, O., Copelli, M.: Optimal dynamical range of excitable networks at criticality. Nature Physics 2(5), 348–351 (2006)
Klyubin, A.S., Polani, D., Nehaniv, C.L.: Tracking Information Flow through the Environment: Simple Cases of Stigmergy. In: Pollack, J., Bedau, M., Husbands, P., Ikegami, T., Watson, R.A. (eds.) Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALife IX), Boston, USA, pp. 563–568. MIT Press, Cambridge (2004)
Klyubin, A.S., Polani, D., Nehaniv, C.L.: All Else Being Equal Be Empowered. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS (LNAI), vol. 3630, pp. 744–753. Springer, Heidelberg (2005)
Lafusa, A., Bossomaier, T.: Hyperplane Localisation of Self-Replicating and Other Complex Cellular Automata Rules. In: Proceedings of the the 2005 IEEE Congress on Evolutionary Computation, Edinburgh, vol. 1, pp. 844–849. IEEE Press (2005)
Langton, C.G.: Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42(1-3), 12–37 (1990)
Lindgren, K., Nordahl, M.G.: Complexity Measures and Cellular Automata. Complex Systems 2(4), 409–440 (1988)
Lindgren, K., Nordahl, M.G.: Universal computation in simple one-dimensional cellular automata. Complex Systems 4, 299–318 (1990)
Lindner, M., Vicente, R., Priesemann, V., Wibral, M.: TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neuroscience 12(1), 119+ (2011)
Lizier, J.T.: JIDT: An information-theoretic toolkit for studying the dynamics of complex systems (2012), https://code.google.com/p/information-dynamics-toolkit/
Lizier, J.T.: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses. Springer, Heidelberg (2013)
Lizier, J.T., Atay, F.M., Jost, J.: Information storage, loop motifs, and clustered structure in complex networks. Physical Review E 86(2), 026110+ (2012a)
Lizier, J.T., Flecker, B., Williams, P.L.: Towards a synergy-based approach to measuring information modification. In: Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), pp. 43–51. IEEE (2013)
Lizier, J.T., Heinzle, J., Horstmann, A., Haynes, J.-D., Prokopenko, M.: Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. Journal of Computational Neuroscience 30(1), 85–107 (2011a)
Lizier, J.T., Mahoney, J.R.: Moving frames of reference, relativity and invariance in transfer entropy and information dynamics. Entropy 15(1), 177–197 (2013)
Lizier, J.T., Pritam, S., Prokopenko, M.: Information dynamics in small-world Boolean networks. Artificial Life 17(4), 293–314 (2011b)
Lizier, J.T., Prokopenko, M.: Differentiating information transfer and causal effect. European Physical Journal B 73(4), 605–615 (2010)
Lizier, J.T., Prokopenko, M., Tanev, I., Zomaya, A.Y.: Emergence of Glider-like Structures in a Modular Robotic System. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, pp. 366–373. MIT Press, Cambridge (2008a)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Detecting Non-trivial Computation in Complex Dynamics. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 895–904. Springer, Heidelberg (2007)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: The information dynamics of phase transitions in random Boolean networks. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, pp. 374–381. MIT Press, Cambridge (2008b)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local information transfer as a spatiotemporal filter for complex systems. Physical Review E 77(2), 026110+ (2008c)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Information modification and particle collisions in distributed computation. Chaos 20(3), 037109+ (2010)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Coherent information structure in complex computation. Theory in Biosciences 131(3), 193–203 (2012b)
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local measures of information storage in complex distributed computation. Information Sciences 208, 39–54 (2012c)
Lungarella, M., Sporns, O.: Mapping information flow in sensorimotor networks. PLoS Computational Biology 2(10), e144+ (2006)
MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)
Marinazzo, D., Wu, G., Pellicoro, M., Angelini, L., Stramaglia, S.: Information flow in networks and the law of diminishing marginal returns: evidence from modeling and human electroencephalographic recordings. PloS ONE 7(9), e45026+ (2012)
Martinez, G.J., Adamatzky, A., McIntosh, H.V.: Phenomenology of glider collisions in cellular automaton Rule 54 and associated logical gates. Chaos, Solitons and Fractals 28(1), 100–111 (2006)
McIntosh, H.V.: Linear Cellular Automata. Universidad Autónoma de Puebla, Puebla, Mexico (1990)
Mitchell, M.: A Complex-Systems Perspective on the “Computation vs. Dynamics” Debate in Cognitive Science. In: Gernsbacher, M.A., Derry, S.J. (eds.) Proceedings of the 20th Annual Conference of the Cognitive Science Society (Cogsci 1998), Madison, Wisconsin, pp. 710–715 (1998a)
Mitchell, M.: Computation in Cellular Automata: A Selected Review. In: Gramss, T., Bornholdt, S., Gross, M., Mitchell, M., Pellizzari, T. (eds.) Non-Standard Computation, pp. 95–140. VCH Verlagsgesellschaft, Weinheim (1998b)
Mitchell, M., Crutchfield, J.P., Das, R.: Evolving Cellular Automata with Genetic Algorithms: A Review of Recent Work. In: Goodman, E.D., Punch, W., Uskov, V. (eds.) Proceedings of the First International Conference on Evolutionary Computation and Its Applications, Moscow, Russia, Russian Academy of Sciences (1996)
Mitchell, M., Crutchfield, J.P., Hraber, P.T.: Evolving Cellular Automata to Perform Computations: Mechanisms and Impediments. Physica D 75, 361–391 (1994)
Morgado, R., Cieśla, M., Longa, L., Oliveira, F.A.: Synchronization in the presence of memory. Europhysics Letters 79(1), 10002 (2007)
Obst, O., Boedecker, J., Schmidt, B., Asada, M.: On active information storage in input-driven systems. arXiv:1303.5526 (2013)
Oxford English Dictionary (2008), http://www.oed.com/ (accessed August 5, 2008)
Pahle, J., Green, A.K., Dixon, C.J., Kummer, U.: Information transfer in signaling pathways: a study using coupled simulated and experimental data. BMC Bioinformatics 9, 139 (2008)
Prokopenko, M., Boschietti, F., Ryan, A.J.: An Information-Theoretic Primer on Complexity, Self-Organization, and Emergence. Complexity 15(1), 11–28 (2009)
Prokopenko, M., Gerasimov, V., Tanev, I.: Evolving Spatiotemporal Coordination in a Modular Robotic System. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 558–569. Springer, Heidelberg (2006)
Prokopenko, M., Lizier, J.T., Obst, O., Wang, X.R.: Relating Fisher information to order parameters. Physical Review E 84, 041116+ (2011)
Prokopenko, M., Lizier, J.T., Price, D.C.: On thermodynamic interpretation of transfer entropy. Entropy 15(2), 524–543 (2013)
Sánchez-Montañés, M.A., Corbacho, F.J.: Towards a New Information Processing Measure for Neural Computation. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 637–642. Springer, Heidelberg (2002)
Schreiber, T.: Measuring Information Transfer. Physical Review Letters 85(2), 461–464 (2000)
Shalizi, C.R.: Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata. PhD thesis, University of Wisconsin-Madison (2001)
Shalizi, C.R., Crutchfield, J.P.: Computational mechanics: Pattern and Prediction, Structure and Simplicity. Journal of Statistical Physics 104, 817–879 (2001)
Shalizi, C.R., Haslinger, R., Rouquier, J.-B., Klinkner, K.L., Moore, C.: Automatic filters for the detection of coherent structure in spatiotemporal systems. Physical Review E 73(3), 036104 (2006)
Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27 (1948)
Takens, F.: Detecting strange attractors in turbulence. In: Rand, D., Young, L.-S. (eds.) Dynamical Systems and Turbulence, Warwick 1980. Lecture Notes in Mathematics, vol. 21, pp. 366–381. Springer, Heidelberg (1981)
Von Neumann, J.: Theory of self-reproducing automata. University of Illinois Press, Urbana (1966)
Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.: Quantifying and Tracing Information Cascades in Swarms. PLoS ONE 7(7), e40084+ (2012)
Wibral, M., Pampu, N., Priesemann, V., Siebenhühner, F., Seiwert, H., Lindner, M., Lizier, J.T., Vicente, R.: Measuring Information-Transfer delays. PLoS ONE 8(2), e55809+ (2013)
Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. Progress in Biophysics and Molecular Biology 105(1-2), 80–97 (2011)
Williams, P.L., Beer, R.D.: Nonnegative Decomposition of Multivariate Information. arXiv:1004.2515 (2010)
Wolfram, S.: Cellular automata as models of complexity. Nature 311(5985), 419–424 (1984a)
Wolfram, S.: Computation theory of cellular automata. Communications in Mathematical Physics 96(1), 15–57 (1984b)
Wolfram, S.: Universality and complexity in cellular automata. Physica D 10(1-2), 1–35 (1984c)
Wolfram, S.: A New Kind of Science. Wolfram Media, Champaign (2002)
Wuensche, A.: Classifying cellular automata automatically: Finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 4(3), 47–66 (1999)
Yamada, T., Aihara, K.: Spatio-temporal complex dynamics and computation in chaotic neural networks. In: Proceedings of the IEEE Symposium on Emerging Technologies and Factory Automation (ETFA 1994), Tokyo, pp. 239–244. IEEE (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lizier, J.T., Prokopenko, M., Zomaya, A.Y. (2014). A Framework for the Local Information Dynamics of Distributed Computation in Complex Systems. In: Prokopenko, M. (eds) Guided Self-Organization: Inception. Emergence, Complexity and Computation, vol 9. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53734-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-53734-9_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53733-2
Online ISBN: 978-3-642-53734-9
eBook Packages: EngineeringEngineering (R0)