Advertisement

Stabilising Hebbian Learning with a Third Factor in a Food Retrieval Task

  • Adedoyin Maria Thompson
  • Bernd Porr
  • Florentin Wörgötter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4095)

Abstract

When neurons fire together they wire together. This is Donald Hebb’s famous postulate. However, Hebbian learning is inherently unstable because synaptic weights will self amplify themselves: the more a synapse is able to drive a postsynaptic cell the more the synaptic weight will grow. We present a new biologically realistic way how to stabilise synaptic weights by introducing a third factor which switches on or off learning so that self amplification is minimised. The third factor can be identified by the activity of dopaminergic neurons in VTA which fire when a reward has been encountered. This leads to a new interpretation of the dopamine signal which goes beyond the classical prediction error hypothesis. The model is tested by a real world task where a robot has to find “food disks” in an environment.

Keywords

Conditioned Stimulus Synaptic Weight Lateral Hypothalamus Real Robot Successful Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hebb, D.O.: The organization of behavior: A neurophychological study. Wiley-Interscience, New York (1949)Google Scholar
  2. 2.
    Kosco, B.: Differential hebbian learning. In: Denker, J.S. (ed.) Neural Networks for computing. AIP conference proceedings, Snowbird, Utah, vol. 151, pp. 277–282. American Institute of Physics, New York (1986)Google Scholar
  3. 3.
    Porr, B., Wörgötter, F.: Isotropic Sequence Order learning. Neural Comp. 15, 831–864 (2003)MATHCrossRefGoogle Scholar
  4. 4.
    Bailey, C.H., Giustetto, M., Huang, Y.Y., Hawkins, R.D., Kandel, E.R.: Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat. Rev. Neurosci. 1(1), 11–20 (2000)CrossRefGoogle Scholar
  5. 5.
    Grossberg, S., Schmajuk, N.: Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Networks 2, 79–102 (1989)CrossRefGoogle Scholar
  6. 6.
    Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425, 620–624 (2003)CrossRefGoogle Scholar
  7. 7.
    Porr, B., Wörgötter, F.: Isotropic sequence order learning in a closed loop behavioural system. Roy. Soc. Phil. Trans. Math., Phys. & Eng. Sciences 361(1811), 2225–2244 (2003)CrossRefGoogle Scholar
  8. 8.
    Ziemke, T.: Are robots embodied? In: First international workshop on epigenetic robotics Modeling Cognitive Development in Robotic Systems, Lund, vol. 85 (2001)Google Scholar
  9. 9.
    Sutton, R.: Learning to predict by method of temporal differences. Machine Learning 3(1), 9–44 (1988)Google Scholar
  10. 10.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2002th edn. Bradford Books, MIT Press (1998)Google Scholar
  11. 11.
    Markram, H., Lübke, J., Frotscher, M., Sakman, B.: Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 275, 213–215 (1997)CrossRefGoogle Scholar
  12. 12.
    Saudargiene, A., Porr, B., Wörgötter, F.: How the shape of pre- and postsynaptic signals can influence stdp: A biophysical model. Neural Comp. 16, 595–626 (2004)MATHCrossRefGoogle Scholar
  13. 13.
    Centonze, D., Picconi, B., Gubellini, P., Bernardi, G., Calabresi, P.: Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur. J. Neurosci. 13(6), 1071–1077 (2001)CrossRefGoogle Scholar
  14. 14.
    Reynolds, J.N., Wickens, J.R.: Dopamine dependent plasticity of corticostriatal synapses. Neural Networks 15, 507–521 (2002)CrossRefGoogle Scholar
  15. 15.
    Wörgötter, F., Porr, B.: Temporal sequence learning, prediction and control - a review of different models and their relation to biological mechanisms. Neural Comp. 17, 245–319 (2005)CrossRefGoogle Scholar
  16. 16.
    Zhang, L.I., Tao, H.W., Holt, C.E., Harris, W.A., Poo, M.: A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37–44 (1998)CrossRefGoogle Scholar
  17. 17.
    Schultz, W.: Dopamine neurons and their role in reward mechanisms. Curr. Opin. Neurobiol. 7(2), 191–197 (1997)CrossRefGoogle Scholar
  18. 18.
    Prescott, T.J., González, F.M.M., Gurney, K., Humpries, M.D., Redgrave, P.: A robot model of the basal ganglia: Behaviour and intrinsic processing. Neural Networks (in press, 2006)Google Scholar
  19. 19.
    Dayan, P., Balleine, B.W.: Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Adedoyin Maria Thompson
    • 1
  • Bernd Porr
    • 1
  • Florentin Wörgötter
    • 2
  1. 1.Department of Electronics & Electrical EngineeringUniversity of GlasgowGlasgowUnited Kingdom
  2. 2.Bernstein Center of Computational NeuroscienceUniversity GöttingenGermany

Personalised recommendations