Neural Processing Letters

, Volume 26, Issue 3, pp 233–249 | Cite as

Generative Modeling of Autonomous Robots and their Environments using Reservoir Computing

  • Eric. A. Antonelo
  • Benjamin Schrauwen
  • Jan Van Campenhout


Autonomous mobile robots form an important research topic in the field of robotics due to their near-term applicability in the real world as domestic service robots. These robots must be designed in an efficient way using training sequences. They need to be aware of their position in the environment and also need to create models of it for deliberative planning. These tasks have to be performed using a limited number of sensors with low accuracy, as well as with a restricted amount of computational power. In this contribution we show that the recently emerged paradigm of Reservoir Computing (RC) is very well suited to solve all of the above mentioned problems, namely learning by example, robot localization, map and path generation. Reservoir Computing is a technique which enables a system to learn any time-invariant filter of the input by training a simple linear regressor that acts on the states of a high-dimensional but random dynamic system excited by the inputs. In addition, RC is a simple technique featuring ease of training, and low computational and memory demands.


Reservoir computing Generative modeling Map learning T-maze task Road sign problem Path generation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Antonelo EA, Baerlvedt A-J, Rognvaldsson T, Figueiredo M (2006) Modular neural network and classical reinforcement learning for autonomous robot navigation: inhibiting undesirable behaviors. In: Proceedings of IJCNN 2006, Vancouver, CanadaGoogle Scholar
  2. 2.
    Antonelo EA, Schrauwen B, Dutoit X, Stroobandt D and Nuttin M (2007). Event detection and localization in mobile robot navigation using reservoir computing. In: (eds) ICANN, Part II, pp 660–669. Springer-Verlag, Berlin Google Scholar
  3. 3.
    Arleo A, Smeraldi F and Gerstner W (2004). Cognitive navigation based on nonuniform gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Trans NN 15(3): 639–652 Google Scholar
  4. 4.
    Bailey T, Durrant-Whyte H (2006) Simultaneous localisation and mapping (SLAM): Part ii state of the art. Rob Autom Mag September 2006Google Scholar
  5. 5.
    Forster A, Graves A, Schmidhuber J (2007) RNN-based learning of compact maps for efficient robot localization. In Proceedings of ESANNGoogle Scholar
  6. 6.
    Jaeger H (2001) Short term memory in echo state networks. Technical Report GMD Report 152, German National Research Center for Information TechnologyGoogle Scholar
  7. 7.
    Jaeger H (2002) Tutorial on training recurrent neural networks, covering bptt, rtrl, ekf and the “echo state network” approach. Technical Report GMD Report 159, German National Research Center for Information TechnologyGoogle Scholar
  8. 8.
    Jaeger H and Haas H (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless telecommunication. Science 308: 78–80 CrossRefADSGoogle Scholar
  9. 9.
    Jaeger H, Lukosevicius M and Popovici D (2007). Optimization and applications of echo state networks with leaky integrator neurons. Neural Netw 20: 335–352 zbMATHCrossRefGoogle Scholar
  10. 10.
    Krose BJA, Eecen M (1994) A self-organizing representation of sensor space for mobile robot navigation. In: Proc. of the IEEE int. conf. on intelligent robots and systems, September 1994Google Scholar
  11. 11.
    Maass W, Natschläger T and Markram H (2002). Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14(11): 2531–2560 zbMATHCrossRefGoogle Scholar
  12. 12.
    Maass W, Joshi P, Sontag E (2006) Principles of real-time computing with feedback applied to cortical microcircuit models. In: Weiss Y, Schölkoff B, Platt J (eds) Advances in neural information processing systems, vol 18. MIT Press, CambridgeGoogle Scholar
  13. 13.
    Najand S, Lo Z, Bavarian B (1992) Applications of self-organizing neural networks for mobile robot environment learning. In: Bekey GA, Goldberg KY (eds), Neural networks in robotics. Kluwer Academic Publishers, pp 85–96Google Scholar
  14. 14.
    Nikolić D, Häusler S, Singer W, Maass W (2007) Temporal dynamics of information content carried by neurons in the primary visual cortex. In: Advances in neural information processing systems 19, vol 19Google Scholar
  15. 15.
    Rylatt RM and Czarnecki CA (2000). Embedding connectionist autonomous agents in time: the ‘road sign problem’. Neural Proc Lett 12: 145–158 zbMATHCrossRefGoogle Scholar
  16. 16.
    Schrauwen B, D’Haene M, Verstraeten D, Van Campenhout J (2007) Compact hardware for real-time speech recognition using a liquid state machine. In: Proceedings of the IJCNNGoogle Scholar
  17. 17.
    Schrauwen B, Verstraeten D, Van Campenhout J (2007) An overview of reservoir computing: theory, applications and implementations. In: Proceedings of the European symposium on artifical neural networks (ESANN)Google Scholar
  18. 18.
    Steil JJ (2004) Backpropagation-decorrelation: online recurrent learning with O(N) complexity. In: Proceedings of IJCNN ’04, vol 1, pp 843–848Google Scholar
  19. 19.
    Thrun S (1998). Learning maps for indoor mobile robot navigation. Artif Intell 99: 21–71 zbMATHCrossRefGoogle Scholar
  20. 20.
    Verstraeten D, Schrauwen B, D’Haene M and Stroobandt D (2007). A unifying comparison of reservoir computing methods. Neural Netw 20: 391–403 zbMATHCrossRefGoogle Scholar
  21. 21.
    Yamazaki T and Tanaka S (2007). The cerebellum as a liquid state machine. Neural Netw 20: 290–297 zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2007

Authors and Affiliations

  • Eric. A. Antonelo
    • 1
  • Benjamin Schrauwen
    • 1
  • Jan Van Campenhout
    • 1
  1. 1.Electronics and Information Systems (ELIS) DepartmentGhent UniversityGhentBelgium

Personalised recommendations