Abstract
We present a biologically inspired recurrent network of spiking neurons and a learning rule that enables the network to balance a ball on a flat circular arena and to steer it towards a target field, by controlling the inclination angles of the arena. The neural controller is a recurrent network of adaptive exponential integrate and fire neurons configured and connected to match properties of cortical layer IV. The network is used as a liquid state machine with four action cells as readout neurons. The solution of the task requires the controller to take its own reaction time into account by anticipating the future state of the controlled system. We demonstrate that the cortical network can robustly learn this task using a supervised learning rule that penalizes the error on the force applied to the arena.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Joshi, P., Maass, W.: Movement Generation with Circuits of Spiking Neurons. Neural Computation 17, 1715–1738 (2005)
Burgsteiner, H.: Training Networks of Biological Realistic Spiking Neurons for Real-Time Robot Control (2005)
Bouganis, A., Shanahan, M.: Training a Spiking Neural Network to Control a 4-DoF Robotic Arm based on Spike Timing-Dependent Plasticity. In: IEEE World Congress on Computational Intelligence, Barcelona, pp. 4104–4111 (2010)
Maass, W., Natschlaeger, T., Markram, H.: Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations. Neural Computation 14(11), 2531–2560 (2002)
Brette, R., Gerstner, W.: Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity. J. Neurophysiol. 94, 3637–3642 (2005)
Gewaltig, M.-O., Diesmann, M.: NEST (Neural Simulation Tool). Scholarpedia 2(4), 1430 (2007)
Nordlie, E., Gewaltig, M.-O., Plesser, H.E.: Towards Reproducible Descriptions of Neuronal Network Models. PLoS Comput. Biol. 5(8), e1000456 (2009)
Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail. PLoS Compuatational Biology 5(12), e1000586 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Probst, D., Maass, W., Markram, H., Gewaltig, MO. (2012). Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7552. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33269-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-33269-2_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33268-5
Online ISBN: 978-3-642-33269-2
eBook Packages: Computer ScienceComputer Science (R0)