Advertisement

Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball

  • Dimitri Probst
  • Wolfgang Maass
  • Henry Markram
  • Marc-Oliver Gewaltig
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7552)

Abstract

We present a biologically inspired recurrent network of spiking neurons and a learning rule that enables the network to balance a ball on a flat circular arena and to steer it towards a target field, by controlling the inclination angles of the arena. The neural controller is a recurrent network of adaptive exponential integrate and fire neurons configured and connected to match properties of cortical layer IV. The network is used as a liquid state machine with four action cells as readout neurons. The solution of the task requires the controller to take its own reaction time into account by anticipating the future state of the controlled system. We demonstrate that the cortical network can robustly learn this task using a supervised learning rule that penalizes the error on the force applied to the arena.

Keywords

Spiking neural networks NEST dynamic control task neurobotics brain-inspired computing AdEx supervised learning closed-loop motor control 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Joshi, P., Maass, W.: Movement Generation with Circuits of Spiking Neurons. Neural Computation 17, 1715–1738 (2005)zbMATHCrossRefGoogle Scholar
  2. 2.
    Burgsteiner, H.: Training Networks of Biological Realistic Spiking Neurons for Real-Time Robot Control (2005)Google Scholar
  3. 3.
    Bouganis, A., Shanahan, M.: Training a Spiking Neural Network to Control a 4-DoF Robotic Arm based on Spike Timing-Dependent Plasticity. In: IEEE World Congress on Computational Intelligence, Barcelona, pp. 4104–4111 (2010)Google Scholar
  4. 4.
    Maass, W., Natschlaeger, T., Markram, H.: Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations. Neural Computation 14(11), 2531–2560 (2002)zbMATHCrossRefGoogle Scholar
  5. 5.
    Brette, R., Gerstner, W.: Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity. J. Neurophysiol. 94, 3637–3642 (2005)CrossRefGoogle Scholar
  6. 6.
    Gewaltig, M.-O., Diesmann, M.: NEST (Neural Simulation Tool). Scholarpedia 2(4), 1430 (2007)CrossRefGoogle Scholar
  7. 7.
    Nordlie, E., Gewaltig, M.-O., Plesser, H.E.: Towards Reproducible Descriptions of Neuronal Network Models. PLoS Comput. Biol. 5(8), e1000456 (2009)Google Scholar
  8. 8.
    Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail. PLoS Compuatational Biology 5(12), e1000586 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dimitri Probst
    • 1
    • 3
  • Wolfgang Maass
    • 2
  • Henry Markram
    • 1
  • Marc-Oliver Gewaltig
    • 1
  1. 1.Blue Brain ProjectÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
  2. 2.Institute for Theoretical Computer ScienceTechnische Universität GrazGrazAustria
  3. 3.Ruprecht-Karls Universität HeidelbergHeidelbergGermany

Personalised recommendations