Skip to main content
SpringerLink
Log in

Information Transfer in Biological and Bio-Inspired Systems

  • Chapter
  • First Online:
The Local Information Dynamics of Distributed Computation in Complex Systems

Part of the book series: Springer Theses ((Springer Theses))

  • 825 Accesses

Abstract

The main thrust of the preceding chapters has been the presentation of a framework for the local information dynamics of computation. Importantly, we have demonstrated that the underlying measures align with our qualitative notions of information storage, transfer and modification in analysing well-understood theoretical systems, including CAs and RBNs. We have also demonstrated that the perspective of local measures, i.e. quantifying the information measures at each point in space and time, produces more detailed insights into a given computation than averaged measures can.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the interregional information networks inferred here are effective networks rather than structural networks [5, 6]. Effective networks consist of a set of directed links representing statistical dependencies between the nodes in an underlying structural network. Effective networks provide insight into the logical structure of the network and how this changes as a function of network activity (regardless of whether the underlying structure is known).

  2. 2.

    As described in Sect. 4.1.3, a collective interaction occurs for example in the XOR operation, where the outcome is not due to the isolated actions of single sources. Collective interactions are captured as interaction-based transfer in the complete and collective TE.

  3. 3.

    The algorithm for inferring directed interregional information structure, and results of its application to fMRI data from a visuomotor tracking task were first reported in [14, 15].

  4. 4.

    Information theory is known to produce useful insights from analysis of fMRI images, e.g. [16, 17]. The novelty here lies in its combination with asymmetric, multivariate and statistical significance-based techniques to infer directed information structure.

  5. 5.

    Functional magnetic resonance imaging (fMRI) is a brain imaging technique which produces a multivariate time-series of measurements for many spatial points (voxels) within the brain at high spatial resolution (typically 3 mm\(^3\) volumes). The measurements are of changes in blood flow and oxygenation related to neural activity near that point.

  6. 6.

    Note the TE could be computed using Kraskov estimation [20, 21] but with a direct conditional MI calculation as per [22, 23].

  7. 7.

    We alter our information-theoretical notation in this section from \(I_{X ; Y}\) to \(I(X ; Y)\) in order to accommodate the complex notation for the variables under consideration.

  8. 8.

    Note the similarity between the multivariate TE and the collective TE  defined in Eq. (4.41). The distinction is that here we have a multivariate destination, whereas the collective measure considers a multivariate source only. This extension eliminates information that was already present in any of the destination variables. This is important for considering regions of variables, where one wishes to eliminate information already contained elsewhere in the region (even if that information moves around the variables within that region). The collective TE focuses on the information added by multiple sources to a single destination only.

  9. 9.

    More specifically, this appears to be a consideration of the collective TE .

  10. 10.

    For example, fMRI regions contain potentially hundreds of voxels, see Table 8.1.

  11. 11.

    The following explanation assumes that only one previous state \(y_n\) of the source is used in the computation of \(T_k(Y \rightarrow X)\); i.e. the parameter \(l=1\) (see Eq. (4.1)).

  12. 12.

    We note the different approach taken to inferring links with the interregional MI in [12], where the authors examined its statistical significance as compared to the average over multivariate MIs computed from random subsets of \(v\) variables taken from any of the brain regions.

  13. 13.

    The fMRI data set was obtained by researchers from the Bernstein Center for Computational Neuroscience in Berlin (see Acknowledgements on p. xx). Furthermore, the data set was subjected by these researchers to standard preprocessing operations (motion correction, spatial normalisation), and a general linear model (GLM) was used to find regions that were activated during the task. The data collection and these prior analyses are described in [30].

  14. 14.

    Figures 8.3 and 8.4 were generated using Cytoscape [31].

  15. 15.

    The application presented in this section was first reported in [38].

  16. 16.

    As we will discuss later in this section, we saw in earlier chapters that the ECAs with the highest average information transfer values did not contain gliders. As such, selection for high transfer could not automatically be expected to result in glider-like coherent structures here.

  17. 17.

    The implementation of and original genetic programming framework for the snakebot were supplied by Ivan Tanev (see Acknowledgements on p. xx).

  18. 18.

    Videos of the snakebot, showing raw motion and local transfer entropy are available at http://lizier.me/joseph/publications/08ALifeSnakebotTe or http://www.prokopenko.net/modular_robotics.html or http://www.youtube.com/view_play_list?p=6604CF436CC0738C

  19. 19.

    We know from the earlier chapters that the ECAs with the largest apparent TE values are not the ECAs exhibiting gliders, so it is unlikely that gliders are equivalent to a maximisation of TE at each separate communications link. We do know that ECAs with gliders exhibit the largest proportion of apparent to complete TE , so this proportion is perhaps related to communication over long distances rather than single communication links.

  20. 20.

    This is effectively the reason that there are very few points of negative local TE measured in the snakebot here; see Fig. 8.7.

References

  1. T. Schreiber, Measuring information transfer. Phys. Rev. Lett. 85(2), 461–464 (2000)

    Article  Google Scholar 

  2. D.R. Rigney, A.L. Goldberger, W. Ocasio, Y. Ichimaru, G.B. Moody, R. Mark, in Multi-channel Physiological Data: description and analysis, ed. by A.S. Weigend, N.A. Gershenfeld. Time Series Prediction: forecasting the future and understanding the past (Addison-Wesley, Reading, 1993), pp. 105–129

    Google Scholar 

  3. M. Rubinov, S.A. Knock, C.J. Stam, S. Micheloyannis, A.W.F. Harris, L.M. Williams, M. Breakspear, Small-world properties of nonlinear brain activity in schizophrenia. Hum. Brain Mapp. 30, 403–416 (2009)

    Article  Google Scholar 

  4. R.A. Stevenson, S. Kim, T.W. James, An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp. Brain Res. 198(2–3), 183–194 (2009)

    Google Scholar 

  5. K.J. Friston, Functional and effective connectivity in neuroimaging: a synthesis. Hum. Brain Mapp. 2, 56–78 (1994)

    Article  Google Scholar 

  6. C.J. Honey, R. Kotter, M. Breakspear, O. Sporns, Network structure of cerebral cortex shapes functional connectivity on multiple time scales. in Proceedings of the National Academy of Sciences, vol. 104, no.24, pp. 10240–10245. 2007

    Google Scholar 

  7. C.S. Soon, M. Brass, H.-J. Heinze, J.-D. Haynes, Unconscious determinants of free decisions in the human brain. Nat. Neurosci. 11(5), 543–545 (2008)

    Article  Google Scholar 

  8. S. Bode, J.-D. Haynes, Decoding sequential stages of task preparation in the human brain. NeuroImage 45(2), 606–613 (2009)

    Article  Google Scholar 

  9. S.L. Bressler, W. Tang, C.M. Sylvester, G.L. Shulman, M. Corbetta, Top-down control of human visual cortex by frontal and parietal cortex in anticipatory visual spatial attention. J. Neurosci. 28(40), 10056–10061 (2008)

    Google Scholar 

  10. A. Tang, C. Honey, J. Hobbs, A. Sher, A. Litke, O. Sporns, J. Beggs, Information flow in local cortical networks is not democratic. BMC Neurosci. 9(Suppl 1), O3 (2008)

    Article  Google Scholar 

  11. H. Hinrichs, H.J. Heinze, M.A. Schoenfeld, Causal visual interactions as revealed by an information theoretic measure and fMRI. NeuroImage 31(3), 1051–1060 (2006)

    Article  Google Scholar 

  12. B. Chai, D. B. Walther, D. M. Beck, L. Fei-Fei, in Exploring Functional Connectivity of the Human Brain Using Multivariate Information Analysis, ed. by Y. Bengio, D. Schuurmans, J. Lafferty, C.K.I. Williams, A. Culottain. Advances in Neural Information Processing Systems, vol. 22, pp. 270–278 (NIPS Foundation, San Diego, 2009)

    Google Scholar 

  13. H. Liang, M. Ding, S.L. Bressler, Temporal dynamics of information flow in the cerebral cortex. Neurocomputing 38–40, 1429–1435 (2001)

    Article  Google Scholar 

  14. J.T. Lizier, J-D. Haynes, J. Heinzle, M. Prokopenko, Directed information structure in inter-regional cortical interactions in a visuomotor tracking task. in Proceedings of the Eighteenth Annual Computational Neuroscience Meeting Computational Neuroscience 2009 (CNS*2009), BMC Neuroscience, 10(Suppl 1) (Germany, Berlin, 2009), p. P117

    Google Scholar 

  15. J.T. Lizier, J. Heinzle, A. Horstmann, J.-D. Haynes, M. Prokopenko, Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. J. Comput. Neurosci. 30(1), 85–107 (2011)

    Article  MathSciNet  Google Scholar 

  16. K. Young, Y. Chen, J. Kornak, G.B. Matson, N. Schuff, Summarizing complexity in high dimensions. Phys. Rev. Lett. 94(9), 098701 (2005)

    Article  Google Scholar 

  17. K. Young, N. Schuff, Measuring structural complexity in brain images. NeuroImage 39(4), 1721–1730 (2008)

    Article  Google Scholar 

  18. K.A. Norman, S.M. Polyn, G.J. Detre, J.V. Haxby, Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cognitive Sci. 10(9), 424–430 (2006)

    Article  Google Scholar 

  19. A.S. Klyubin, D. Polani, C.L. Nehaniv, Representations of space and time in the maximization of information flow in the perception-action loop. Neural Comput. 19(9), 2387–2432 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. A. Kraskov, H. Stögbauer, P. Grassberger, Estimating mutual information. Phys. Rev. E 69(6), 066138 (2004)

    Article  MathSciNet  Google Scholar 

  21. A. Kraskov, Synchronization and Interdependence Measures and their Applications to the Electroencephalogram of Epilepsy Patients and Clustering of Data. Ph.D. thesis, ser. Publication Series of the John von Neumann Institute for Computing, vol. 24 (John von Neumann Institute for Computing, Jülich, 2004)

    Google Scholar 

  22. S. Frenzel, B. Pompe, Partial mutual information for coupling analysis of multivariate time series. Phys. Rev. Lett. 99(20), 204101 (2007)

    Article  Google Scholar 

  23. G. Gomez-Herrero, W. Wu, K. Rutanen, M.C. Soriano, G. Pipa, R. Vicente, Assessing coupling dynamics from an ensemble of time series (2010), arXiv:1008.0539, http://arxiv.org/abs/1008.0539. Accessed 2010

  24. M. Chávez, J. Martinerie, M. Le Van Quyen, Statistical assessment of nonlinear causality: application to epileptic EEG signals. J. Neurosci. Methods 124(2), 113–128 (2003)

    Article  Google Scholar 

  25. M. Grosse-Wentrup, in Understanding Brain Connectivity Patterns During Motor Imagery for Brain-Computer Interfacing, ed by D. Koller, D. Schuurmans, Y. Bengio, L. Bottou. Advances in Neural Information Processing Systems, vol. 21 (Curran Associates, New York, 2008), pp. 561–568

    Google Scholar 

  26. T.Q. Tung, T. Ryu, K.H. Lee, D. Lee, Inferring gene regulatory networks from microarray time series data using transfer entropy, ed. by P. Kokol, V. Podgorelec, D. Mičetič-Turk, M. Zorman, M. Verlič. in Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS’07), Maribor, Slovenia (IEEE, Los Alamitos, USA, 2007), pp. 383–388

    Google Scholar 

  27. J.V. Haxby, M.I. Gobbini, M.L. Furey, A. Ishai, J.L. Schouten, P. Pietrini, Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293(5539), 2425–2430 (2001)

    Article  Google Scholar 

  28. M. Lungarella, T. Pegors, D. Bulwinkle, O. Sporns, Methods for quantifying the informational structure of sensory and motor data. Neuroinformatics 3(3), 243–262 (2005)

    Article  Google Scholar 

  29. P.F. Verdes, Assessing causality from multivariate time series. Phys. Rev. E 72(2), 026 222–0262229 (2005)

    Google Scholar 

  30. A. Horstmann, Sensorimotor integration in human eye-hand coordination: neuronal correlates and characteristics of the system, Ph.D. Dissertation (Ruhr-Universität Bochum, Bochum, 2008)

    Google Scholar 

  31. P. Shannon, A. Markiel, O. Ozier, N.S. Baliga, J.T. Wang, D. Ramage, N. Amin, B. Schwikowski, T. Ideker, Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 13(11), 2498–2504 (2003)

    Article  Google Scholar 

  32. P. Gong, C. van Leeuwen, Distributed dynamical computation in neural circuits with propagating coherent activity patterns. PLoS Comput. Biol. 5(12), e1000611 (2009)

    Article  Google Scholar 

  33. M. Prokopenko, V. Gerasimov, I. Tanev, Evolving spatiotemporal coordination in a modular robotic system, ed. by S. Nolfi, G. Baldassarre, R. Calabretta, J. Hallam, D. Marocco, J.-A. Meyer, D. Parisiin. Proceedings of the Ninth International Conference on the Simulation of Adaptive Behavior (SAB’06), Rome, ser. Lecture Notes in Artificial Intelligence, vol. 4095 (Springer, Heidelberg, 2006), pp. 548–559

    Google Scholar 

  34. D. Polani, O. Sporns, M. Lungarella, How information and embodiment shape intelligent information processing, ed. by M. Lungarella, F. Iida, J. Bongard, R. Pfeifer. in Proceedings of the 50th Anniversary Summit of Artificial Intelligence, New York, ser. Lecture Notes in Computer Science, vol. 4850 (Springer, Berlin, 2007), pp. 99–111

    Google Scholar 

  35. A.S. Klyubin, D. Polani, C.L. Nehaniv, All else being equal be empowered, ed. by M.S. Capcarrere, A.A. Freitas, P.J. Bentley, C.G. Johnson, J. Timmis. in Proceedings of the 8th European Conference on Artificial Life (ECAL), Kent, UK, ser. Lecture Notes in Computer Science, vol. 3630 (Springer, Heidelberg, 2005), pp. 744–753

    Google Scholar 

  36. O. Sporns, M. Lungarella, Evolving coordinated behavior by maximizing information structure, ed. by L.M. Rocha, L.S. Yaeger, M.A. Bedau, D. Floreano, R.L. Goldstone, A. Vespignani. in Proceedings of the Tenth International Conference on Simulation and Synthesis of Living Systems (ALifeX), Bloomington, Indiana, USA (MIT Press, Cambridge, 2006), pp. 323–329

    Google Scholar 

  37. N. Ay, N. Bertschinger, R. Der, F. Güttler, E. Olbrich, Predictive information and explorative behavior of autonomous robots. Eur. Phys. J. B 63(3), 329–339 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  38. J.T. Lizier, M. Prokopenko, I. Tanev, A.Y. Zomaya, Emergence of glider-like structures in a modular robotic system, ed. by S. Bullock, J. Noble, R. Watson, M.A. Bedau. in Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK (MIT Press, Cambridge, 2008), pp. 366–373

    Google Scholar 

  39. M. Prokopenko, V. Gerasimov, I. Tanev, Measuring spatiotemporal coordination in a modular robotic system ed. by L.M. Rocha, L.S. Yaeger, M.A. Bedau, D. Floreano, R.L. Goldstone, A. Vespignani. in Proceedings of the 10th International Conference on the Simulation and Synthesis of Living Systems (ALifeX), Bloomington, Indiana, USA (MIT Press, Cambridge, 2006), pp. 185–191

    Google Scholar 

  40. I. Tanev, T. Ray, A. Buller, Automated evolutionary design, robustness, and adaptation of sidewinding locomotion of a simulated snake-like robot. IEEE Trans. Robot. 21(4), 632–645 (2005)

    Article  Google Scholar 

  41. J.T. Lizier, M. Prokopenko, A.Y. Zomaya, Local information transfer as a spatiotemporal filter for complex systems. Phys. Rev. E 77(2), 026110 (2008)

    Article  MathSciNet  Google Scholar 

  42. J.A. Brown, J.A. Tuszynski, A review of the ferroelectric model of microtubules. Ferroelectrics 220, 141–156 (1999)

    Article  Google Scholar 

  43. I. Couzin, R. James, D. Croft, J. Krause, in Social Organization and Information Transfer in Schooling Fishes, ed by B.C.K. Laland, J. Krause. Fish Cognition and Behavior, ser. Fish and Aquatic Resources (Blackwell Publishing, Oxford, 2006), pp. 166–185

    Google Scholar 

  44. M. Mitchell, J.P. Crutchfield, P.T. Hraber, Evolving cellular automata to perform computations: mechanisms and impediments. Physica D 75, 361–391 (1994)

    Article  MATH  Google Scholar 

  45. M. Mitchell, J.P. Crutchfield, R. Das, Evolving cellular automata with genetic algorithms: a review of recent work ed. by E.D. Goodman, W. Punch, V. Uskov. in Proceedings of the First International Conference on Evolutionary Computation and Its Applications, Moscow (Russian Academy of Sciences, Russia, 1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CSIRO ICT Centre, Corner of Vimiera and Pembroke Road, Marsfield, NSW, 2122, Australia

    Joseph T. Lizier

Authors
  1. Joseph T. Lizier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph T. Lizier .

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Lizier, J.T. (2013). Information Transfer in Biological and Bio-Inspired Systems. In: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32952-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32952-4_8

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32951-7

  • Online ISBN: 978-3-642-32952-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation