Distributed Adaptive Control: A Proposal on the Neuronal Organization of Adaptive Goal Oriented Behavior

  • Armin Duff
  • César Rennó-Costa
  • Encarni Marcos
  • Andre L. Luvizotto
  • Andrea Giovannucci
  • Marti Sanchez-Fibla
  • Ulysses Bernardet
  • Paul F. M. J. Verschure

Abstract

In behavioral motor coordination and interaction it is a fundamental challenge how an agent can learn to perceive and act in unknown and dynamic environments. At present, it is not clear how an agent can – without any explicitly predefined knowledge – acquire internal representations of the world while interacting with the environment. To meet this challenge, we propose a biologically based cognitive architecture called Distributed Adaptive Control (DAC). DAC is organized in three different, tightly coupled, layers of control: reactive, adaptive and contextual. DAC based systems are self-contained and fully grounded, meaning that they autonomously generate representations of their primary sensory inputs, hence bootstrapping their behavior form simple to advance interactions. Following this approach, we have previously identified a novel environmentally mediated feedback loop in the organization of perception and behavior, i.e. behavioral feedback. Additionally, we could demonstrated that the dynamics of the memory structure of DAC, acquired during a foraging task, are equivalent to a Bayesian description of foraging. In this chapter we present DAC in a concise form and show how it is allowing us to extend the different subsystems to more biophysical detailed models. These further developments of the DAC architecture, not only allow to better understand the biological systems, but moreover advance DACs behavioral capabilities and generality.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004)CrossRefGoogle Scholar
  2. 2.
    Bermudez i Badia, S., Pyk, P., Verschure, P.F.M.J.: A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance. Int. J. Robot Res. 26, 759–772 (2007)CrossRefGoogle Scholar
  3. 3.
    Bakin, J.S., Weinberger, N.M.: Classical conditioning induces CS-specific receptive field plasticity in the auditory cortex of the guinea pig. Brain Res. 536(1-2), 271–286 (1990)CrossRefGoogle Scholar
  4. 4.
    Bayes, M., Price, M.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R Soc. London 53, 370–418 (1763)Google Scholar
  5. 5.
    Becker, S., Plumbley, M.: Unsupervised neural network learning procedures for feature extraction and classification. Appl. Intell. 6(3), 185–203 (1996)CrossRefGoogle Scholar
  6. 6.
    Bell, A.J.: Levels and loops: the future of artificial intelligence and neuroscience. Philos. Trans. R Soc. Lond B Biol. Sci. 354(1392), 2013–2020 (1999)CrossRefGoogle Scholar
  7. 7.
    Berlau, K.M., Weinberger, N.M.: Learning strategy determines auditory cortical plasticity. Neurobiol. Learn. Mem. 89(2), 153–166 (2008)CrossRefGoogle Scholar
  8. 8.
    Bernardet, U.: The neurobiological basis of perception and behavior: the iqr large-scale neuronal system simulator and its application. Ph.D. thesis, University of Zurich (2007)Google Scholar
  9. 9.
    Bernardet, U., Bermúdez i Badia, S., Verschure, P.F.M.J.: A model for the neuronal substrate of dead reckoning and memory in arthropods: a comparative computational and behavioral study. Theory Biosci. 127(2) (2008)Google Scholar
  10. 10.
    Braitenberg, V.: Vehicles, experiments in synthetic psychology. MIT Press, Cambridge (1984)Google Scholar
  11. 11.
    Brooks, R.: Intelligence without representation. Artif. Intell. 47(991), 139–159 (1991)CrossRefGoogle Scholar
  12. 12.
    Brooks, R.: New approaches to robotics. Science 253(5025), 1227–1232 (1991)CrossRefGoogle Scholar
  13. 13.
    Clancey, W.: Situated Cognition: On human knowledge and computer representations. Cambridge University Press, Cambridge (1996)Google Scholar
  14. 14.
    Davis, H.: Underestimating the rat’s intelligence. Brain Res. Cogn. Brain Res. 3(3-4), 291–298 (1996)CrossRefGoogle Scholar
  15. 15.
    Duff, A., Wyss, R., Verschure, P.F.M.J.: Learning temporally stable representations from natural sounds: Temporal stability as a general objective underlying sensory processing. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4669, pp. 129–138. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Edeline, J.M.: Learning-induced physiological plasticity in the thalamo-cortical sensory systems: a critical evaluation of receptive field plasticity, map changes and their potential mechanisms. Prog. Neurobiol. 57(2), 165–224 (1999)CrossRefGoogle Scholar
  17. 17.
    Edeline, J.M., Pham, P., Weinberger, N.M.: Rapid development of learning-induced receptive field plasticity in the auditory cortex. Behav. Neurosci. 107(4), 539–551 (1993)CrossRefGoogle Scholar
  18. 18.
    Foldiak, P.: Learning invariance from transformation sequences. Neural Comput. 3(2), 194–200 (1991)CrossRefGoogle Scholar
  19. 19.
    Franzius, M., Sprekeler, H., Wiskott, L.: Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Comput. Biol. 3(8), e166 (2007)Google Scholar
  20. 20.
    Gallistel, C.R.: The Organization of Learning. MIT Press, Cambridge (1990)Google Scholar
  21. 21.
    Galvan, V.V., Weinberger, N.M.: Long-term consolidation and retention of learning-induced tuning plasticity in the auditory cortex of the guinea pig. Neurobiol. Learn. Mem. 77(1), 78–108 (2002)CrossRefGoogle Scholar
  22. 22.
    Georgopoulos, A.: New concepts in generation of movement. Neuron 13, 257–268 (1994)CrossRefGoogle Scholar
  23. 23.
    Gibson, J.J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum, New Jersey (1979)Google Scholar
  24. 24.
    Harnad, S.: The symbol grounding problem. Physica D 42(1-3), 335–346 (1990)CrossRefGoogle Scholar
  25. 25.
    Herbort, O., Butz, M.V., Pedersen, G.: The sure reach model for motor learning and control of a redundant arm: from modeling human behavior to applications in robots. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 85–106. Springer, Heidelberg (2010)Google Scholar
  26. 26.
    Hipp, J., Einhäuser, W., Conradt, J., König, P.: Learning of somatosensory representations for texture discrimination using a temporal coherence principle. Network 16(2-3), 223–238 (2005)CrossRefGoogle Scholar
  27. 27.
    Hofstötter, C., Mintz, M., Verschure, P.F.M.J.: The cerebellum in action: a simulation and robotics study. Eur. J. Neurosci. 16(7), 1361–1376 (2002)CrossRefGoogle Scholar
  28. 28.
    Homberg, U.: In search of the sky compass in the insect brain. Naturwissenschaften 91, 199–208 (2004)CrossRefGoogle Scholar
  29. 29.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U S A 79(8), 2554–2558 (1982)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Hoyer, P.O., Hyvärinen, A.: A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 42(12), 1593–1605 (2002)CrossRefGoogle Scholar
  31. 31.
    Hurri, J., Hyvärinen, A.: Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput. 15(3), 663–691 (2003)MATHCrossRefGoogle Scholar
  32. 32.
    Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. J. Wiley, New York (2001)CrossRefGoogle Scholar
  33. 33.
    Klein, D.J., König, P., Körding, K.P.: Sparse spectrotemporal coding of sounds. Eurasip Jasp 3, 659–667 (2003)Google Scholar
  34. 34.
    Konorski, J.: Integrative activity of the brain: An interdisciplinary approach. University of Chicago Press, Chicago (1967)Google Scholar
  35. 35.
    Körding, K.P., Kayser, C., Einhäuser, W., König, P.: How are complex cell properties adapted to the statistics of natural stimuli? J. Neurophysiol. 91(1), 206–212 (2004)CrossRefGoogle Scholar
  36. 36.
    Laird, J.: Using a computer game to develop advanced AI. Computer 34(7), 70–75 (2001)CrossRefGoogle Scholar
  37. 37.
    Lewicki, M.S.: Efficient coding of natural sounds. Nat. Neurosci. 5(4), 356–363 (2002)CrossRefGoogle Scholar
  38. 38.
    MacDonall, J.S., Goodell, J., Juliano, A.: Momentary maximizing and optimal foraging theories of performance on concurrent VR schedules. Behav. Processes 72(3), 283–299 (2006)CrossRefGoogle Scholar
  39. 39.
    Mackintosh, N.J.: Conditioning and associative learning. Oxford psychology series. Clarendon Press, Oxford (1990) (Reprint)Google Scholar
  40. 40.
    Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., Doucet, A.: A Bayesian Exploration-Exploitation Approach for Optimal Online Sensing and Planning with a Visually Guided Mobile Robot. Auton Robots (in press, 2009)Google Scholar
  41. 41.
    McCarthy, J., Hayes, P.J.: Some philosophical problems from the standpoint of artificial intelligence. Mach. Intell. 4, 463–502 (1969)MATHGoogle Scholar
  42. 42.
    Mitrovic, D., Klanke, S., Vijayakumar, S.: Adaptive optimal feedback control with learned internal dynamics models. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 65–83. Springer, Heidelberg (2010)Google Scholar
  43. 43.
    Montemerlo, M., Thrun, S.: FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics (Springer Tracts in Advanced Robotics). Springer, New York (2007)MATHGoogle Scholar
  44. 44.
    Newell, A.: Unified theories of cognition. Harvard University Press, Cambridge (1990)Google Scholar
  45. 45.
    Ohl, F.W., Scheich, H.: Learning-induced plasticity in animal and human auditory cortex. Curr. Opin. Neurobiol. 15(4), 470–477 (2005)CrossRefGoogle Scholar
  46. 46.
    Oja, E.: A simplified neuron model as a principal component analyzer. J. Math. Biol. 15(3), 267–273 (1982)MATHCrossRefMathSciNetGoogle Scholar
  47. 47.
    Oja, E., Ogawa, H., Wangviwattana, J.: Principal component analysis by homogeneous neural networks, Part I: The weighted subspace criterion. IEICE Trans. Inf. Syst. 75, 366–375 (1992)Google Scholar
  48. 48.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)CrossRefGoogle Scholar
  49. 49.
    O’Regan, J.K., Noe, A.: A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24(5), 939–973 (2001)CrossRefGoogle Scholar
  50. 50.
    Pavlov, I.P.: Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Oxford University Press, Oxford (1927)Google Scholar
  51. 51.
    Pfeifer, R., Scheier, C.: From perception to action: the right direction? In: Gaussier, P., Nicoud, J. (eds.) From Perception to Action Conference, 1994, Proceedings, Los Alamitos, California, pp. 1–11 (1994)Google Scholar
  52. 52.
    Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT Press, Cambridge (1999)Google Scholar
  53. 53.
    Rao, R., Olshausen, B., Lewicki, M.: Probabilistic Models of the Brain: Perception and Neural Function. MIT Press, Cambridge (2002)Google Scholar
  54. 54.
    Rescorla, R., Wagner, A.: A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black, A., Prokasy, W.F. (eds.) Classical Conditioning II: Current Research and Theory, pp. 64–99. Appleton Century Crofts, New York (1972)Google Scholar
  55. 55.
    Roberts, W.: Foraging by rats on a radial maze:learning, memory, and decision rules. In: Gormezano, I., Wasserman, E. (eds.) Learning and memory: The behavioral and biological substrates, pp. 7–24. Lawrence Erlbaum, New Jersey (1992)Google Scholar
  56. 56.
    Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)CrossRefMathSciNetGoogle Scholar
  57. 57.
    Rutkowski, R.G., Weinberger, N.M.: Encoding of learned importance of sound by magnitude of representational area in primary auditory cortex. Proc. Natl. Acad. Sci. U S A 102(38), 13664–13669 (2005)CrossRefGoogle Scholar
  58. 58.
    Sanchez-Montanes, M.A., König, P., Verschure, P.F.M.J.: Learning sensory maps with real-world stimuli in real time using a biophysically realistic learning rule. IEEE Trans. Neural Netw. 13(3), 619–632 (2002)CrossRefGoogle Scholar
  59. 59.
    Schultz, W.: Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006)CrossRefGoogle Scholar
  60. 60.
    Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001)CrossRefGoogle Scholar
  61. 61.
    Smith, E.C., Lewicki, M.S.: Efficient auditory coding. Nature 439(7079), 978–982 (2006)CrossRefGoogle Scholar
  62. 62.
    Sporns, O., Kötter, R.: Motifs in brain networks. PLoS Biol. 2(11), e369 (2004)CrossRefGoogle Scholar
  63. 63.
    Squire, L.R., Kandel, E.R.: Memory: From mind to molecules. Scientific American Library, New York (1999)Google Scholar
  64. 64.
    Sur, M., Leamey, C.A.: Development and plasticity of cortical areas and networks. Nat. Rev. Neurosci. 2(4), 251–262 (2001)CrossRefGoogle Scholar
  65. 65.
    Sutton, R., Barto, A.G.: Reinforcement learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  66. 66.
    Thorndike, E.: Animal Intelligence. Macmillan, New York (1911)Google Scholar
  67. 67.
    Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381(6582), 520–522 (1996)CrossRefGoogle Scholar
  68. 68.
    Tovee, M.J., Rolls, E.T., Treves, A., Bellis, R.P.: Information encoding and the responses of single neurons in the primate temporal visual cortex. J. Neurophysiol. 70(2), 640–654 (1993)Google Scholar
  69. 69.
    Tversky, A., Slovic, B., Kahneman, B.: Judgment under uncertainty: heuristics and biases. Cambridge University Press, Cambridge (2001)Google Scholar
  70. 70.
    Varela, F., Thompson, E., Rosch, E.: The Embodied Mind: Cognitive Science and Human Experience. MIT Press, Cambridge (1991)Google Scholar
  71. 71.
    Vernon, D., Metta, G., Sandini, G.: A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents. IEEE Trans. Evol. Comput. 11(2), 151–180 (2007)CrossRefGoogle Scholar
  72. 72.
    Verschure, P., Mintz, M.: A real-time model of the cerebellar circuitry underlying classical conditioning: A combined simulation and robotics study. Neurocomputing 38(40), 1019–1024 (2001)CrossRefGoogle Scholar
  73. 73.
    Verschure, P.F.M.J.: Synthetic epistemology: The acquisition, retention, and expression of knowledge in natural and synthetic systems. In: IEEE World Conference on Computational Intelligence, Proceedings, pp. 147–152. Anchorage, Alaska (1998)Google Scholar
  74. 74.
    Verschure, P.F.M.J., Althaus, P.: A real-world rational agent: Unifying old and new AI. Cogn. Sci. 27, 561–590 (2003)CrossRefGoogle Scholar
  75. 75.
    Verschure, P.F.M.J., Coolen, A.C.C.: Adaptive fields: distributed representations of classically conditioned associations. Network 2(2), 189–206 (1991)CrossRefGoogle Scholar
  76. 76.
    Verschure, P.F.M.J., Krose, B., Pfeifer, R.: Distributed adaptive control: The self-organization of structured behavior. Rob. Auton. Syst. 9, 181–196 (1993)CrossRefGoogle Scholar
  77. 77.
    Verschure, P.F.M.J., Pfeifer, R.: Categorization, representations, and the dynamics of system-environment interaction: a case study in autonomous systems. In: Meyer, J.A., Roitblat, H., Wilson, S. (eds.) From Animals to Animats: Proceedings of the Second International Conference on Simulation of Adaptive behavior, Honolulu, Hawaii, pp. 210–217. MIT Press, Cambridge (1992)Google Scholar
  78. 78.
    Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425(6958), 620–624 (2003)CrossRefGoogle Scholar
  79. 79.
    Voegtlin, T., Verschure, P.F.M.J.: What can robots tell us about brains? A synthetic approach towards the study of learning and problem solving. Rev. Neurosci. 10(3-4), 291–310 (1999)Google Scholar
  80. 80.
    Wallis, G.: Using spatio-temporal correlations to learn invariant object recognition. Neural Netw. 9(9), 1513–1519 (1996)CrossRefGoogle Scholar
  81. 81.
    Weinberger, N.M.: Learning-induced changes of auditory receptive fields. Curr. Opin. Neurobiol. 3(4), 570–577 (1993)CrossRefGoogle Scholar
  82. 82.
    Weinberger, N.M.: Physiological memory in primary auditory cortex: characteristics and mechanisms. Neurobiol. Learn. Mem. 70(1-2), 226–251 (1998)CrossRefGoogle Scholar
  83. 83.
    Wiskott, L., Sejnowski, T.J.: Slow feature analysis: unsupervised learning of invariances. Neural Comput. 14(4), 715–770 (2002)MATHCrossRefGoogle Scholar
  84. 84.
    Wyss, R.: Sensory and motor coding in the organization of behavior. Ph.D. thesis, ETHZ (2003)Google Scholar
  85. 85.
    Wyss, R., König, P., Verschure, P.F.M.J.: Invariant representations of visual patterns in a temporal population code. Proc. Natl. Acad. Sci. U S A 100(1), 324–329 (2003)CrossRefGoogle Scholar
  86. 86.
    Wyss, R., König, P., Verschure, P.F.M.J.: A model of the ventral visual system based on temporal stability and local memory. PLoS Biol. 4(5), e120 (2006)Google Scholar
  87. 87.
    Wyss, R., Verschure, P.F.M.J., Konig, P.: Properties of a temporal population code. Rev. Neurosci. 14(1-2), 21–33 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Armin Duff
    • 1
  • César Rennó-Costa
    • 3
  • Encarni Marcos
    • 3
  • Andre L. Luvizotto
    • 3
  • Andrea Giovannucci
    • 3
  • Marti Sanchez-Fibla
    • 3
  • Ulysses Bernardet
    • 3
  • Paul F. M. J. Verschure
    • 2
  1. 1.INI Institute of NeuroinformaticsUNI-ETH ZürichZürichSwitzerland
  2. 2.ICREA Institució Catalana de Recerca i Estudis AvançatsBarcelonaSpain
  3. 3.SPECS, IUA, Technology DepartmentUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations