, Volume 3, Issue 3, pp 243–262 | Cite as

Methods for quantifying the informational structure of sensory and motor data

  • Max Lungarella
  • Teresa Pegors
  • Daniel Bulwinkle
  • Olaf SpornsEmail author
Original Article


Embodied agents (organisms and robots) are situated in specific environments sampled by their sensors and within which they carry out motor activity. Their control architectures or nervous systems attend to and process streams of sensory stimulation, and ultimately generate sequences of motor actions, which in turn affect the selection of information. Thus, sensory input and motor activity are continuously and dynamically coupled with the surrounding environment. In this article, we propose that the ability of embodied agents to actively structure their sensory input and to generate statistical regularities represents a major functional rationale for the dynamic coupling between sensory and motor systems. Statistical regularities in the multimodal sensory data relayed to the brain are critical for enabling appropriate developmental processes, perceptual categorization, adaptation, and learning. To characterize the informational structure of sensory and motor data, we introduce and illustrate a set of univariate and multivariate statistical measures (available in an accompanying Matlab toolbox). We show how such measures can be used to quantify the information structure in sensory and motor channels of a robot capable of saliency-based attentional behavior, and discuss their potential importance for understanding sensorimotor coordination in organisms and for robot design.

Index Entries

Sensorimotor coordination embodiment information self-structuring 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alexander, W. and Sporns, O. (2003) An embodied model of reward conditioning. Adaptive Behav. 11(2), 43–159.Google Scholar
  2. Almassy, N., Edelman, G. M., and Sporns, O. (1998) Behavioral constraints in the development of neuronal properties: A cortical model embedded in a real world device. Cereb. Cortex 8, 346–361.PubMedCrossRefGoogle Scholar
  3. Barlow, H. B. (1961) Possible principles underlying the transformation of sensory messages. In: Sensory Communication. Rosenblith W. A. (ed.) MIT Press, Cambridge, MA, pp. 217–234.Google Scholar
  4. Barrow, H. G. (1987) Learning receptive fields. Proc. of First Ann. Conf. on Neural Networks, vol. 15. pp. 115–121.Google Scholar
  5. Barlow, H. B. (2001) The exploitation of regularities in the environment by the brain. Behav. Brain Sci. 24, 602–607.PubMedCrossRefGoogle Scholar
  6. Bell, A. J. and Sejnowski, T. J. (1997) The independent components of natural scenes are edge filters. Vision Res. 37, 3327–3338.PubMedCrossRefGoogle Scholar
  7. Betsch, B. Y., Einhauser, W., Kording, K. P., and Konig, P. (2004) The world from a cat’s perspective—statistics of natural videos. Biol. Cybern. 90, 41–50.PubMedCrossRefGoogle Scholar
  8. Bishop, C. M. (1995) Neural Networks for Pattern Recognition. Clarendon Press, Oxford.Google Scholar
  9. De Silva, V. (2004) Personal communication.Google Scholar
  10. Dong, D. W. and Atick, J. J. (1995) Statistics of natural time-varying images. Netw. Comput. Neural Syst. 6(3), 345–358.CrossRefGoogle Scholar
  11. Field, D. J. (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394.PubMedCrossRefGoogle Scholar
  12. Field, D. J. (1999) Wavelets, vision and the statistics of natural scenes. Phil. Trans. R. Soc. Lond. A857, 2527–2542.CrossRefGoogle Scholar
  13. Grassberger, P. (1988) Finite sample corrections to entropy and dimension estimates. Phys. Lett. A 128, 369.CrossRefGoogle Scholar
  14. Hamilton, J. (1994) Time Series Analysis. Princeton University Press, Princeton, NJ, USA.Google Scholar
  15. Iida, F., Pfeifer, R., Steels, L., and Kuniyoshi, Y. (eds.) (2004) Embodied Artifical Intelligence, LNCS 3139. Springer-Verlag, Heidelberg.Google Scholar
  16. Isliker, H. and Kurths, J. (1993) A test for stationarity: finding parts in time series apt for correlation dimensions estimate. Int. J. Bifurcation Chaos 3, 1573–1579.CrossRefGoogle Scholar
  17. Itti, L., Koch, C., and Niebur, E. (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259.CrossRefGoogle Scholar
  18. Karklin, Y. and Lewicki, M. S. (2003) Learning higher-order structures in natural images. Netw. Comput. Neural Syst. 14, 483–499.CrossRefGoogle Scholar
  19. Kramer, M. (1991) Nonlinear principal component analysis using autoassociative neural networks. AlChE J. 37, 233–243.Google Scholar
  20. Krichmar, J. L., Seth, A. K., Nitz, D. A., Fleischer, J. G., and Edelman, G. M. (2005) Spatial navigation and causal analysis in a brain-based device having detailed cortical-hippocampal interactions. Neuroinformatics 3(3).Google Scholar
  21. Lichtensteiger, L. and Pfeifer, R. (2002) An optimal sensory morphology improves adaptability of neural network controllers. In: Dorronsoro J. R. (ed.) Proc. of Int. Conf. on Artificial Neural Networks. LNCS 2451. Springer-Verlag, Heidelberg, pp. 850–855.Google Scholar
  22. Lungarella, M. and Pfeifer, R. (2001) Robots as cognitive tools: Information-theoretic analysis of sensory-motor data. Proc. of First Int. Conf. on Humanoid Robots, pp. 245–252.Google Scholar
  23. Lungarella, M. and Sporns, O. (2005) Information self-structuring: key principle for development and learning. Proc. Fourth Int. Conf. Devel. Learning, in press.Google Scholar
  24. Metta, G. and Fitzpatrick, P. (2003) Early integration of vision and manipulation. Adaptive Behav. 11(2), 109–128.CrossRefGoogle Scholar
  25. Nolfi, S. (2002) Power and limit of reactive agents. Neurocomputing 49, 119–145.CrossRefGoogle Scholar
  26. Olshausen, B. and Field, D. J. (1996) Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature 381, 607–609.PubMedCrossRefGoogle Scholar
  27. Olsson, L. A., Nehaniv, C. L., and Polani, D. (2004) The effects on visual information in a robot in environments with oriented contours. In: Proceedings of the Fourth International Workshop on Epigenetic Robotics. Metta, G. and Berthouze, L. (eds.) pp. 83–88.Google Scholar
  28. Pfeifer, R. and Scheier, C. (1999) Understanding Intelligence. MIT Press, Cambridge, MA.Google Scholar
  29. Rieke, F., Bodnar, D. A., and Bialek, W. (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc. R. Soc. Lond. B 262, 259–265.CrossRefGoogle Scholar
  30. Roulston, M. S. (1999) Estimating the errors on measured entropy and mutual information. Physica D 125, 285–294.CrossRefGoogle Scholar
  31. Scheier, C., Pfeifer, R., and Kuniyoshi, Y. (1998) Embedded neural networks: exploiting constraints. Neural Netw. 11(7/8), 1551–1569.PubMedCrossRefGoogle Scholar
  32. Schreiber, T. (1997) Detecting and analyzing nonstationarity in a time series using nonlinear cross-prediction. Phys. Rev. Lett. 78(5), 843–847.CrossRefGoogle Scholar
  33. Seth, A. K. and Edelman, G. M. (2004) Environment and behavior influence the complexity of evolved neural networks. Adaptive Behav. 12(1), 5–20.CrossRefGoogle Scholar
  34. Seth, A. K. Causal connectivity of evolved neural networks during behavior. Netw. Comput. Neural Syst., in press.Google Scholar
  35. Shannon, C. E. and Weaver, W. (1949) The Mathematical Theory of Communication. University of Illinois Press, Chicago, IL, USA.Google Scholar
  36. Silverman, B. W. (1986) Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.Google Scholar
  37. Simoncelli, E. and Olshausen, B. (2001) Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216.PubMedCrossRefGoogle Scholar
  38. Simonoff, J. S. (1996) Smoothing Methods in Statistics. Springer-Verlag, Heidelberg.Google Scholar
  39. Sporns, O., Tononi, G., and Edelman, G. M. (2000) Theoretical neuroanatomy: relating anatomical and functional connectivity in graphs and cortical connection matrices. Cereb. Cortex 10, 127–141.PubMedCrossRefGoogle Scholar
  40. Sporns, O. and Pegors, T. K. (2003) Generating structure in sensory data through coordinated motor activity. Proc. of Int. Conf. on Neural Networks 2796.Google Scholar
  41. Sporns, O. and Pegors, T. K. (2004) Information-theoretic aspects of embodied artificial intelligence. In: Embodied Artificial Intelligence, LNCS 3139. Springer-Verlag, Heidelberg. Iida, F., Pfeifer, R., Steels, L., and Kuniyoshi, Y. (eds.) pp. 74–85.Google Scholar
  42. Tarapore, D., Lungarella, M., and Gomez, G. (2005) Quantifying patterns of agent-environment interaction. Robotics and Autonomous Systems (in press).Google Scholar
  43. Te Boekhorst, R., Lungarella, M., and Pfeifer, R. (2003) Dimensionality reduction through sensory-motor coordination. In: Proc. of Joint. Int. Conf. on Artificial Neural Networks and Neural Information Processing, LNCS. 2714, Kanyak, O., Alpaydin, E., Oja, E., and Xu, L. (eds.) pp. 496–503.Google Scholar
  44. Tenenbaum, J., de Silva, V., and Langford, J. (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323.PubMedCrossRefGoogle Scholar
  45. Thelen, E. and Smith, L. (1994) A Dynamic Systems Approach to the Development of Cognition and Action. MIT Press, Cambridge, MA, USA.Google Scholar
  46. Tononi, G., Sporns, O., and Edelman, G. M. (1994) A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc. Natl. Acad. Sci. USA 91, 5033–5037.PubMedCrossRefGoogle Scholar
  47. Tononi, G., Sporns, O., and Edelman, G. M. (1996) A complexity measure for selective matching of signals by the brain. Proc. Natl. Acad. Sci. USA 93, 3422–3427.PubMedCrossRefGoogle Scholar
  48. Tononi, G., Edelman, G. M., and Sporns, O. (1998) Complexity and coherency: integrating information in the brain. Trends Cognitive Sci. 2, 474–484.CrossRefGoogle Scholar
  49. Torralba, A. and Oliva, A. (2003) Statistics of natural images. Netw. Comput. Neural Syst. 14, 391–412.CrossRefGoogle Scholar
  50. van der Schaaf, A. and van Hateren, J. H. (1996) Modeling the power spectra of natural images: Statistics and information. Vision Res. 36(17), 2759–2770.PubMedCrossRefGoogle Scholar
  51. Webb, A. (1999) Statistical Pattern Recognition. Arnold, London.Google Scholar
  52. Webber, C. J. (1991) Competitive learning, natural images, and the self-organization of cortical cells. Netw. Comput. Neural Syst. 2, 169–187.CrossRefGoogle Scholar
  53. Yu, C. and Ballard, D. H. (2004) A multimodal learning interface for grounding spoken language in sensory perceptions. ACM Trans. Appl. Perceptions 1(1), 57–80.CrossRefGoogle Scholar

Copyright information

© Humana Press Inc 2005

Authors and Affiliations

  • Max Lungarella
    • 1
  • Teresa Pegors
    • 2
  • Daniel Bulwinkle
    • 2
  • Olaf Sporns
    • 2
    Email author
  1. 1.Department of Mechano-Informatics, School of Information Science and TechnologyUniversity of TokyoTokyoJapan
  2. 2.Department of PsychologyIndiana UniversityBloomingtonUSA

Personalised recommendations