Skip to main content
SpringerLink
Log in

Information Modification

  • Chapter
  • First Online:
The Local Information Dynamics of Distributed Computation in Complex Systems

Part of the book series: Springer Theses ((Springer Theses))

  • 800 Accesses

Abstract

Information modification is often colloquially described as processing. It has been viewed as a particularly important operation for biological neural networks and models thereof [1–4], where it has been suggested as a potential biological driver [2]. It is also a key operation in collision-based computing (e.g. [5, 6], including soliton dynamics and collisions [7]).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The number of particles resulting from a collision has been studied elsewhere [9].

  2. 2.

    The separable information and its application to CAs were first reported in [15, 16].

  3. 3.

    Interactions are accounted in that equation because of the incremental conditioning on previous sources (i.e. in conditional and complete transfer entropies). Consideration of apparent transfer entropies only in Eq. (5.2)means that interactions are not being accounted for.

  4. 4.

    The separable information has parallels to the sum of “first order terms” (the apparent contribution of each source without considering interactions) in the total information of a destination in [17]. The local separable information however is distinguished in considering the contributions in the context of the destination’s past, and evaluating these on a local scale—both features are critical for an understanding of distributed computation.

  5. 5.

    Only then does the active information storage go negative, because from a temporal perspective the inner domain no longer continues. The separable information is positive here though, as the transfer sources provide positive information about their domains intruding (having coalesced).

  6. 6.

    \(T\) is the absolute temperature and \(k\) is Boltzmann’s constant.

  7. 7.

    Maroney [25] argues that while a logically irreversible transformation of information does generate this amount of heat, it can in fact be accomplished by a thermodynamically reversible mechanism.

  8. 8.

    For example, the result of an annihilation could be trivially reproduced without any of the particles existing in the first place.

  9. 9.

    With this formulation we see that the approach to information processing in [3] (referred to at the start of this chapter) has aspects more akin to destroying redundant information than information modification. Indeed the authors describe part of the approach as measuring “minimization of spurious information” retained in the output.

  10. 10.

    This can be because these past states are direct causal sources of the future state, or are indirectly causal via other variables (such as neighbouring cells in a CA, i.e. as per Sect. 3.1), or perhaps both the past states and future state have a common causal driver.

  11. 11.

    Note the important distinction here. The statistical complexity—excess entropy interpretation computes the information that will be destroyed in total about the current underlying \(\epsilon \)-machine state, over an arbitrary number of time steps. The interpretation in Eq. (5.18) computes the information destroyed about the current underlying state at the next state transition \(x_{n}^{(k^+)} \rightarrow x_{n+1}^{(k^+)}\) only. The distinction parallels that between the excess entropy and active information storage discussed in Chap. 3, and experimental results from the two views could be contrasted in future work.

  12. 12.

    Assumption 1: that all causal inputs to the computation are considered to be within the system, i.e. it is causally closed or closed to efficient cause [35] (which is stronger than informational closure [36]). Assumption 2: (the normal case where) only the state at time step \(n\) is a direct causal input to the state at time step \(n+1\). Under these assumptions, none of the conditions described in footnote 10 apply to the system as a whole, and so the future state \(\mathbf x _{n+1}\) is statistically independent of multiple past states \( \{ \mathbf{x }_{n-c} \mid k \ge c > 0\}\) given the previous state \(\mathbf x _n\) (i.e. \(I\left( \mathbf x _{n-c} ; \mathbf x _{n+1} \mid \mathbf x _n \right) = 0, \forall k \ge c > 0\)). Note that our assumption of causal closure allows for stochasticity in the computation of the next state of the system, but not for correlations across time in such stochasticity.

  13. 13.

    This is akin to using the next state of the whole system \(\mathbf x _{n+1}\) to compute pre-images of the whole system \(\mathbf x _{n}\), by gradually building the previous state cell by cell [37].

  14. 14.

    Note the similarly large local information destruction in the unperturbed domain wall to the left of the particle annihilation. These values are an artifact of \(m=8\) not being large enough to see beyond the large number of consecutive “0” states in the white triangle in Fig. 5.5a to gauge the phase of the surrounding domains.

  15. 15.

    There are a few points around the collision with small non-zero \(d(i,n,m=8)\) up to 0.07 bits. These appear to be largely artifacts of the surrounding area of the system (\(m=8\)) analysed, partially due to the occurrence of similar next state configurations with different preceding states in rare dynamics, rather than being due to irreversibility in the collision itself.

References

  1. O. Kinouchi, M. Copelli, Optimal dynamical range of excitable networks at criticality. Nat. Phys. 2(5), 348–351 (2006)

    Article  Google Scholar 

  2. J.J. Atick, Could information theory provide an ecological theory of sensory processing? Netw. Comput. Neural Syst. 3(2), 213 (1992)

    Google Scholar 

  3. M.A.Sánchez-Montañés, F.J. Corbacho, Towards a new information processing measure for neural computation, ed. by J. Dorronsoroin. Proceedings of the International Conference on Artificial Neural Networks (ICANN 2002), Madrid, Spain. ser. Lecture Notes in Computer Science, vol. 2415 (Berlin/Heidelberg: Springer-Verlag, 2002), pp. 637–642

    Google Scholar 

  4. T. Yamada, K. Aihara, Spatio-temporal complex dynamics and computation in chaotic neural networks, in Proceedings of the IEEE Symposium on Emerging Technologies and Factory Automation (ETFA ’94), Tokyo. IEEE, 1994, pp. 239–244

    Google Scholar 

  5. M.H. Jakubowski, K. Steiglitz, R. Squier, Information transfer between solitary waves in the saturable Schrödinger equation. Phys. Rev. E 56(6), 7267 (1997)

    Article  Google Scholar 

  6. A. Adamatzky (ed.), Collision-Based Computing (Springer-Verlag, Berlin, 2002)

    Google Scholar 

  7. D.E. Edmundson, R.H. Enns, Fully 3-dimensional collisions of bistable light bullets. Opt. Lett. 18, 1609–1611 (1993)

    Article  Google Scholar 

  8. C.G. Langton, Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42(1–3), 12–37 (1990)

    Google Scholar 

  9. W. Hordijk, C.R. Shalizi, J.P. Crutchfield, Upper bound on the products of particle interactions in cellular automata. Physica D 154(3–4), 240–258 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. N. Boccara, J. Nasser, M. Roger, Particlelike structures and their interactions in spatiotemporal patterns generated by one-dimensional deterministic cellular-automaton rules. Phys. Rev. A 44(2), 866–875 (1991)

    Article  Google Scholar 

  11. B. Martin, A group interpretation of particles generated by one dimensional cellular automaton, Wolfram’s 54 rule. Int. J. Mod. Phys. C 11(1), 101–123 (2000)

    Article  MATH  Google Scholar 

  12. G.J. Martinez, A. Adamatzky, H.V. McIntosh, Phenomenology of glider collisions in cellular automaton rule 54 and associated logical gates. Chaos, Solitons Fractals 28(1), 100–111 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. M. Mitchell, J.P. Crutchfield, P.T. Hraber, Evolving cellular automata to perform computations: mechanisms and impediments. Physica D 75, 361–391 (1994)

    Article  MATH  Google Scholar 

  14. M. Mitchell, J.P. Crutchfield, R. Das, Evolving cellular automata with genetic algorithms: a review of recent work, ed. by E.D. Goodman, W. Punch, V. Uskov. in Proceedings of the First International Conference on Evolutionary Computation and Its Applications, Moscow, Russia: Russian Academy of Sciences, 1996

    Google Scholar 

  15. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Detecting non-trivial computation in complex dynamics, ed. by F. Almeida e Costa, L.M. Rocha, E. Costa, I. Harvey, A. Coutinho, in Proceedings of the 9th European Conference on Artificial Life (ECAL), Lisbon, Portugal, ser. Lecture Notes in Artificial Intelligence, vol. 4648. (Springer, Berlin/Heidelberg, 2007), pp. 895–904

    Google Scholar 

  16. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Information modification and particle collisions in distributed computation. Chaos, 20(3), 037109 (2010)

    Google Scholar 

  17. N. Lüdtke, S. Panzeri, M. Brown, D.S. Broomhead, J. Knowles, M.A. Montemurro, D.B. Kell, Information-theoretic sensitivity analysis: a general method for credit assignment in complex networks. J. Roy. Soc. Interface 5(19), 223–235 (2008)

    Article  Google Scholar 

  18. M. Prokopenko, F. Boschietti, A.J. Ryan, An information-theoretic primer on complexity, self-organization, and emergence. Complexity 15(1), 11–28 (2009)

    Article  MathSciNet  Google Scholar 

  19. D.G. Green, Emergent behavior in biological systems. Complex. Int. 1, (1994), paper ID: green01

    Google Scholar 

  20. C.R. Shalizi, R. Haslinger, J.-B. Rouquier, K.L. Klinkner, C. Moore, Automatic filters for the detection of coherent structure in spatiotemporal systems. Phys. Rev. E 73(3), 036104 (2006)

    Article  MathSciNet  Google Scholar 

  21. R. Landauer, Irreversibility and heat generation in the computing process. IBM J. Res. Devel. 5, 183–191 (1961)

    Article  MathSciNet  MATH  Google Scholar 

  22. C.H. Bennett, Notes on Landauer’s principle, reversible computation, and Maxwell’s Demon. Stud. Hist. Philos. Sci. Part B 34(3), 501–510 (2003)

    Article  MATH  Google Scholar 

  23. B. Piechocinska, Information erasure. Phys. Rev. A 61(6), 062314 (2000)

    Article  MathSciNet  Google Scholar 

  24. S. Lloyd, Programming the Universe (Vintage Books, New York, 2006)

    Google Scholar 

  25. O.J.E. Maroney, Generalizing Landauer’s principle. Phys. Rev. E 79(3), 031105 (2009)

    Article  MathSciNet  Google Scholar 

  26. A.W. Burks, On backwards-deterministic, erasable, and Garden-of-Eden automata, Computer and Communication Sciences Department, The University of Michigan, Technical Report 012520–4-T, 1971

    Google Scholar 

  27. D. Richardson, Tessellations with local transformations. J. Comput. Syst. Sci. 6(5), 373–388 (1972)

    Article  MATH  Google Scholar 

  28. T. Toffoli, N.H. Margolus, Invertible cellular automata: a review. Physica D 45(1–3), 229–253 (1990)

    Google Scholar 

  29. T. Helvik, K. Lindgren, M.G. Nordahl, Continuity of information transport in surjective cellular automata. Commun. Math. Phys. 272(1), 53–74 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  30. C. Seife, Decoding the universe (Penguin Group, New York, 2006)

    Google Scholar 

  31. K. Wiesner, M. Gu, E. Rieper, V. Vedral, Information erasure lurking behind measures of complexity, 2009, arXiv:0905.2918v1. Available: http://www.arxiv.org/abs/0905.2918

  32. D.P. Feldman, J.P. Crutchfield, Synchronizing to periodicity: the transient information and synchronization time of periodic sequences. Adv. Complex Syst. 7(3–4), 329–355 (2004)

    Google Scholar 

  33. F. Takens, in Detecting strange attractors in turbulence, ed. by D. Rand, L.-S. Young. Dynamical Systems and Turbulence, Warwick, ser. Lecture Notes in Mathematics, (Springer, Berlin/Heidelberg, 1981), pp. 366–381 (1980)

    Google Scholar 

  34. J.R. Mahoney, C.J. Ellison, J.P. Crutchfield, Information accessibility and cryptic processes. J. Phys. A 42(36), 362002 (2009)

    Article  MathSciNet  Google Scholar 

  35. R. Rosen, Life Itself: A Comprehensive Enquiry into the Nature, Origin and Fabrication of Life (Columbia University Press, New York, 1991)

    Google Scholar 

  36. N. Bertschinger, E. Olbrich, N. Ay, J. Jost, Information and closure in systems theory, ed. by S. Artmann, P. Dittrich in Proceedings of the 7th German Workshop on Artificial Life (GWAL-7), Jena, Germany (IOS Press, Amsterdam, 2006)

    Google Scholar 

  37. A. Wuensche, Classifying cellular automata automatically: finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 4(3), 47–66 (1999)

    Article  MathSciNet  Google Scholar 

  38. B. Misra, I. Prigogine, Irreversibility and nonlocality. Lett. Math. Phys. 7(5), 421–429 (1983)

    Article  MathSciNet  Google Scholar 

  39. C.R. Shalizi, Causal architecture, complexity and self-organization in time series and cellular automata, Ph.D. dissertation, University of Wisconsin-Madison, 2001

    Google Scholar 

  40. C.R. Shalizi, K.L. Shalizi, R. Haslinger, Quantifying self-organization with optimal predictors. Phys. Rev. Lett. 93(11), 118701 (2004)

    Article  Google Scholar 

  41. D. Halliday, R. Resnick, J. Walker, Fundamentals of Physics (Wiley, New York, 1993)

    Google Scholar 

  42. P. Grassberger, Some more exact enumeration results for 1d cellular automata. J. Phys. A: Math. Gen. 20(12), 4039–4046 (1987)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CSIRO ICT Centre, Corner of Vimiera and Pembroke Rd, Marsfield, 2122, Australia

    Joseph T. Lizier

Authors
  1. Joseph T. Lizier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph T. Lizier .

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Lizier, J.T. (2013). Information Modification. In: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32952-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32952-4_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32951-7

  • Online ISBN: 978-3-642-32952-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation