Information Theoretic Self-organised Adaptation in Reservoirs for Temporal Memory Tasks
Recurrent neural networks of the Reservoir Computing (RC) type have been found useful in various time-series processing tasks with inherent non-linearity and requirements of temporal memory. Here with the aim to obtain extended temporal memory in generic delayed response tasks, we combine a generalised intrinsic plasticity mechanism with an information storage based neuron leak adaptation rule in a self-organised manner. This results in adaptation of neuron local memory in terms of leakage along with inherent homeostatic stability. Experimental results on two benchmark tasks confirm the extended performance of this system as compared to a static RC and RC with only intrinsic plasticity. Furthermore, we demonstrate the ability of the system to solve long temporal memory tasks via a simulated T-shaped maze navigation scenario.
KeywordsRecurrent neural networks Self-adaptation Information theory Intrinsic plasticity
Unable to display preview. Download preview PDF.
- 5.Antonelo, E., Schrauwen, B., Stroobandt, D.: Mobile Robot Control in the Road Sign Problem using Reservoir Computing Networks. In: Proceedings of the IEEE Int. Conf. on Robotics and Automation, ICRA (2008)Google Scholar
- 11.Jaeger, H.: Adaptive Nonlinear System Identification with Echo State Networks. In: Advances in Neural Information Processing Systems, vol. 15 (2003)Google Scholar