Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball
We present a biologically inspired recurrent network of spiking neurons and a learning rule that enables the network to balance a ball on a flat circular arena and to steer it towards a target field, by controlling the inclination angles of the arena. The neural controller is a recurrent network of adaptive exponential integrate and fire neurons configured and connected to match properties of cortical layer IV. The network is used as a liquid state machine with four action cells as readout neurons. The solution of the task requires the controller to take its own reaction time into account by anticipating the future state of the controlled system. We demonstrate that the cortical network can robustly learn this task using a supervised learning rule that penalizes the error on the force applied to the arena.
KeywordsSpiking neural networks NEST dynamic control task neurobotics brain-inspired computing AdEx supervised learning closed-loop motor control
Unable to display preview. Download preview PDF.
- 2.Burgsteiner, H.: Training Networks of Biological Realistic Spiking Neurons for Real-Time Robot Control (2005)Google Scholar
- 3.Bouganis, A., Shanahan, M.: Training a Spiking Neural Network to Control a 4-DoF Robotic Arm based on Spike Timing-Dependent Plasticity. In: IEEE World Congress on Computational Intelligence, Barcelona, pp. 4104–4111 (2010)Google Scholar
- 7.Nordlie, E., Gewaltig, M.-O., Plesser, H.E.: Towards Reproducible Descriptions of Neuronal Network Models. PLoS Comput. Biol. 5(8), e1000456 (2009)Google Scholar
- 8.Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail. PLoS Compuatational Biology 5(12), e1000586 (2009)Google Scholar