Abstract
Embodied agents (organisms and robots) are situated in specific environments sampled by their sensors and within which they carry out motor activity. Their control architectures or nervous systems attend to and process streams of sensory stimulation, and ultimately generate sequences of motor actions, which in turn affect the selection of information. Thus, sensory input and motor activity are continuously and dynamically coupled with the surrounding environment. In this article, we propose that the ability of embodied agents to actively structure their sensory input and to generate statistical regularities represents a major functional rationale for the dynamic coupling between sensory and motor systems. Statistical regularities in the multimodal sensory data relayed to the brain are critical for enabling appropriate developmental processes, perceptual categorization, adaptation, and learning. To characterize the informational structure of sensory and motor data, we introduce and illustrate a set of univariate and multivariate statistical measures (available in an accompanying Matlab toolbox). We show how such measures can be used to quantify the information structure in sensory and motor channels of a robot capable of saliency-based attentional behavior, and discuss their potential importance for understanding sensorimotor coordination in organisms and for robot design.
Similar content being viewed by others
References
Alexander, W. and Sporns, O. (2003) An embodied model of reward conditioning. Adaptive Behav. 11(2), 43–159.
Almassy, N., Edelman, G. M., and Sporns, O. (1998) Behavioral constraints in the development of neuronal properties: A cortical model embedded in a real world device. Cereb. Cortex 8, 346–361.
Barlow, H. B. (1961) Possible principles underlying the transformation of sensory messages. In: Sensory Communication. Rosenblith W. A. (ed.) MIT Press, Cambridge, MA, pp. 217–234.
Barrow, H. G. (1987) Learning receptive fields. Proc. of First Ann. Conf. on Neural Networks, vol. 15. pp. 115–121.
Barlow, H. B. (2001) The exploitation of regularities in the environment by the brain. Behav. Brain Sci. 24, 602–607.
Bell, A. J. and Sejnowski, T. J. (1997) The independent components of natural scenes are edge filters. Vision Res. 37, 3327–3338.
Betsch, B. Y., Einhauser, W., Kording, K. P., and Konig, P. (2004) The world from a cat’s perspective—statistics of natural videos. Biol. Cybern. 90, 41–50.
Bishop, C. M. (1995) Neural Networks for Pattern Recognition. Clarendon Press, Oxford.
De Silva, V. (2004) Personal communication.
Dong, D. W. and Atick, J. J. (1995) Statistics of natural time-varying images. Netw. Comput. Neural Syst. 6(3), 345–358.
Field, D. J. (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394.
Field, D. J. (1999) Wavelets, vision and the statistics of natural scenes. Phil. Trans. R. Soc. Lond. A857, 2527–2542.
Grassberger, P. (1988) Finite sample corrections to entropy and dimension estimates. Phys. Lett. A 128, 369.
Hamilton, J. (1994) Time Series Analysis. Princeton University Press, Princeton, NJ, USA.
Iida, F., Pfeifer, R., Steels, L., and Kuniyoshi, Y. (eds.) (2004) Embodied Artifical Intelligence, LNCS 3139. Springer-Verlag, Heidelberg.
Isliker, H. and Kurths, J. (1993) A test for stationarity: finding parts in time series apt for correlation dimensions estimate. Int. J. Bifurcation Chaos 3, 1573–1579.
Itti, L., Koch, C., and Niebur, E. (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259.
Karklin, Y. and Lewicki, M. S. (2003) Learning higher-order structures in natural images. Netw. Comput. Neural Syst. 14, 483–499.
Kramer, M. (1991) Nonlinear principal component analysis using autoassociative neural networks. AlChE J. 37, 233–243.
Krichmar, J. L., Seth, A. K., Nitz, D. A., Fleischer, J. G., and Edelman, G. M. (2005) Spatial navigation and causal analysis in a brain-based device having detailed cortical-hippocampal interactions. Neuroinformatics 3(3).
Lichtensteiger, L. and Pfeifer, R. (2002) An optimal sensory morphology improves adaptability of neural network controllers. In: Dorronsoro J. R. (ed.) Proc. of Int. Conf. on Artificial Neural Networks. LNCS 2451. Springer-Verlag, Heidelberg, pp. 850–855.
Lungarella, M. and Pfeifer, R. (2001) Robots as cognitive tools: Information-theoretic analysis of sensory-motor data. Proc. of First Int. Conf. on Humanoid Robots, pp. 245–252.
Lungarella, M. and Sporns, O. (2005) Information self-structuring: key principle for development and learning. Proc. Fourth Int. Conf. Devel. Learning, in press.
Metta, G. and Fitzpatrick, P. (2003) Early integration of vision and manipulation. Adaptive Behav. 11(2), 109–128.
Nolfi, S. (2002) Power and limit of reactive agents. Neurocomputing 49, 119–145.
Olshausen, B. and Field, D. J. (1996) Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature 381, 607–609.
Olsson, L. A., Nehaniv, C. L., and Polani, D. (2004) The effects on visual information in a robot in environments with oriented contours. In: Proceedings of the Fourth International Workshop on Epigenetic Robotics. Metta, G. and Berthouze, L. (eds.) pp. 83–88.
Pfeifer, R. and Scheier, C. (1999) Understanding Intelligence. MIT Press, Cambridge, MA.
Rieke, F., Bodnar, D. A., and Bialek, W. (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc. R. Soc. Lond. B 262, 259–265.
Roulston, M. S. (1999) Estimating the errors on measured entropy and mutual information. Physica D 125, 285–294.
Scheier, C., Pfeifer, R., and Kuniyoshi, Y. (1998) Embedded neural networks: exploiting constraints. Neural Netw. 11(7/8), 1551–1569.
Schreiber, T. (1997) Detecting and analyzing nonstationarity in a time series using nonlinear cross-prediction. Phys. Rev. Lett. 78(5), 843–847.
Seth, A. K. and Edelman, G. M. (2004) Environment and behavior influence the complexity of evolved neural networks. Adaptive Behav. 12(1), 5–20.
Seth, A. K. Causal connectivity of evolved neural networks during behavior. Netw. Comput. Neural Syst., in press.
Shannon, C. E. and Weaver, W. (1949) The Mathematical Theory of Communication. University of Illinois Press, Chicago, IL, USA.
Silverman, B. W. (1986) Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
Simoncelli, E. and Olshausen, B. (2001) Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216.
Simonoff, J. S. (1996) Smoothing Methods in Statistics. Springer-Verlag, Heidelberg.
Sporns, O., Tononi, G., and Edelman, G. M. (2000) Theoretical neuroanatomy: relating anatomical and functional connectivity in graphs and cortical connection matrices. Cereb. Cortex 10, 127–141.
Sporns, O. and Pegors, T. K. (2003) Generating structure in sensory data through coordinated motor activity. Proc. of Int. Conf. on Neural Networks 2796.
Sporns, O. and Pegors, T. K. (2004) Information-theoretic aspects of embodied artificial intelligence. In: Embodied Artificial Intelligence, LNCS 3139. Springer-Verlag, Heidelberg. Iida, F., Pfeifer, R., Steels, L., and Kuniyoshi, Y. (eds.) pp. 74–85.
Tarapore, D., Lungarella, M., and Gomez, G. (2005) Quantifying patterns of agent-environment interaction. Robotics and Autonomous Systems (in press).
Te Boekhorst, R., Lungarella, M., and Pfeifer, R. (2003) Dimensionality reduction through sensory-motor coordination. In: Proc. of Joint. Int. Conf. on Artificial Neural Networks and Neural Information Processing, LNCS. 2714, Kanyak, O., Alpaydin, E., Oja, E., and Xu, L. (eds.) pp. 496–503.
Tenenbaum, J., de Silva, V., and Langford, J. (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323.
Thelen, E. and Smith, L. (1994) A Dynamic Systems Approach to the Development of Cognition and Action. MIT Press, Cambridge, MA, USA.
Tononi, G., Sporns, O., and Edelman, G. M. (1994) A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc. Natl. Acad. Sci. USA 91, 5033–5037.
Tononi, G., Sporns, O., and Edelman, G. M. (1996) A complexity measure for selective matching of signals by the brain. Proc. Natl. Acad. Sci. USA 93, 3422–3427.
Tononi, G., Edelman, G. M., and Sporns, O. (1998) Complexity and coherency: integrating information in the brain. Trends Cognitive Sci. 2, 474–484.
Torralba, A. and Oliva, A. (2003) Statistics of natural images. Netw. Comput. Neural Syst. 14, 391–412.
van der Schaaf, A. and van Hateren, J. H. (1996) Modeling the power spectra of natural images: Statistics and information. Vision Res. 36(17), 2759–2770.
Webb, A. (1999) Statistical Pattern Recognition. Arnold, London.
Webber, C. J. (1991) Competitive learning, natural images, and the self-organization of cortical cells. Netw. Comput. Neural Syst. 2, 169–187.
Yu, C. and Ballard, D. H. (2004) A multimodal learning interface for grounding spoken language in sensory perceptions. ACM Trans. Appl. Perceptions 1(1), 57–80.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lungarella, M., Pegors, T., Bulwinkle, D. et al. Methods for quantifying the informational structure of sensory and motor data. Neuroinform 3, 243–262 (2005). https://doi.org/10.1385/NI:3:3:243
Issue Date:
DOI: https://doi.org/10.1385/NI:3:3:243